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Why  GAO  Did  This  Study 

As  part  of  its  efforts  to  meet  yearly 
recruitment  goals  for  the  military,  DOD 
requested  almost  $575  million  in  fiscal 
year  2017  to  conduct  advertising 
intended  to  increase  awareness  of 
military  service  and  ultimately  generate 
leads  for  potential  recruits. 

Senate  Report  114-49  included  a 
provision  for  GAO  to  assess  DOD’s 
advertising  activities.  This  report 
examines  the  extent  to  which  DOD 
(1)  has  coordinated  advertising 
activities  among  the  military  service 
components,  (2)  has  followed 
commercial  best  practices  to  assess 
the  effectiveness  of  advertising 
activities,  and  (3)  has  oversight  of  its 
components’  advertising  activities. 

GAO  identified  best  practices  for 
assessing  the  effectiveness  of 
advertising  in  consultation  with 
advertising  industry  experts  and 
reviewed  DOD  and  service  policies 
and  the  most  recent  version  of  each 
military  service  components’ 
advertising  plans,  from  fiscal  years 
2014-16. 

What  GAO  Recommends 

GAO  recommends  that  DOD  develop  a 
formal  process  for  coordination  among 
its  components  on  crosscutting 
advertising  issues;  ensure  that  the 
components  fully  measure  the 
performance  of  advertising  activities; 
and  issue  department-wide  policy  that, 
among  other  things,  defines  DOD’s 
oversight  role  in  advertising.  DOD 
generally  agreed  with  GAO’s 
recommendations. 


View  GAO-16-396.  For  more  information, 
contact  Andrew  Von  Ah  at  (213)  830-101 1  or 
vonaha@gao.gov. 
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What  GAO  Found 

The  Department  of  Defense  (DOD)  has  taken  steps  to  coordinate  some 
advertising  activities  among  the  military  service  components,  but  it  has  not 
developed  a  formal  process  for  coordination.  DOD  conducts  joint  market 
research  and  service  officials  responsible  for  advertising  at  times  share  some 
information  about  lessons  learned.  However,  there  is  no  formal  process  for 
addressing  inefficiencies  and  to  ensure  information  sharing  among  the  services. 
GAO  found  examples  of  possible  unnecessary  duplication,  overlap,  and 
fragmentation  that  may  result  from  the  absence  of  coordination.  For  example,  the 
Air  Force  has  three  advertising  programs  that  contract  with  three  advertising 
agencies,  but  officials  could  not  provide  a  rationale  for  requiring  separate 
programs.  In  the  absence  of  a  formal  process  for  coordination,  the  services  may 
be  missing  opportunities  to  effectively  leverage  advertising  resources. 

While  DOD  has  generally  followed  commercial  best  practices  GAO  identified  to 
assess  the  effectiveness  of  advertising,  DOD's  components  vary  in  their  ability  to 
determine  whether  their  activities  are  generating  leads  for  potential  recruits.  For 
example,  while  the  Marine  Corps  has  developed  a  framework  to  assess  the 
effectiveness  of  its  advertising  including  leads  generated  from  advertising 
activities  at  the  local  level,  Army  officials  stated  they  do  not  have  reliable  data  to 
evaluate  whether  locally  executed  advertising  activities  are  generating  leads,  and 
the  Army  National  Guard  does  not  require  state  units  to  report  on  the 
performance  of  their  advertising  activities.  Without  fully  measuring  advertising 
performance,  especially  at  the  local  levels,  DOD  may  be  unable  to  ensure 
advertising  dollars  are  used  efficiently  to  help  meet  recruiting  goals. 

DOD  does  not  have  comprehensive  oversight  of  the  components’  advertising 
activities;  instead,  DOD's  components  oversee  their  own  programs.  However, 
examples  identified  by  GAO  and  others  of  some  components  paying  sport  teams 
to  provide  recognition  ceremonies  for  servicemembers — a  practice  later  deemed 
unacceptable  by  DOD — suggest  that  the  absence  of  DOD  oversight  may  have 
contributed  to  some  activities  of  questionable  appropriateness.  Further,  GAO 
observed  discrepancies  in  how  recent  sports  advertising  guidance  was  being 
interpreted  and  in  service  officials’  understanding  of  regulations  that  direct  digital 
advertising.  Without  a  department-wide  policy  that  clearly  defines  its  oversight 
role,  DOD  lacks  reasonable  assurance  that  advertising  is  carried  out  in  an 
appropriate  manner. 


Examples  of  military  service  social  media  advertising,  mobile  assets  for  advertising,  and  print 
advertising. 
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Washington,  DC  20548 


U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


May  12,  2016 

The  Honorable  John  McCain 
Chairman 

The  Honorable  Jack  Reed 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Mac  Thornberry 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

Each  year,  the  Department  of  Defense  (DOD)  seeks  to  meet  recruitment 
goals  targeting  the  small  percentage  of  the  U.S.  population  between  the 
ages  of  17  and  24  that,  according  to  DOD  officials,  is  able  to  meet  the 
education  and  physical  fitness  standards  required  for  military  service. 
According  to  DOD  officials,  advertising  is  one  of  several  tools,  which  also 
includes  recruiters,  that  the  department  uses  to  influence  individuals  to 
consider  military  service.  DOD  requested  almost  $575  million  for  fiscal 
year  2017  for  its  advertising  programs.  Advertising  is  the  placement  of 
messages  intended  to  inform  or  persuade  an  audience  through  various 
types  of  media,  such  as  television,  radio,  digital  media,  direct  mail,  and 
others.  Sports-related  advertising  is  among  the  strategies  that  the  military 
services  have  increasingly  used  to  reach  their  target  market  for 
advertising.  Each  military  service  carries  out  advertising  activities  to  raise 
the  public’s  awareness  of  the  military  and  help  recruiters  generate  leads 
of  potential  recruits.  Public  and  congressional  interest  in  DOD  advertising 
increased  after  2015  media  reports  that  an  Army  National  Guard  unit  had 
paid  a  professional  football  team  to  conduct  ceremonies  that  recognized 
or  honored  servicemembers,  a  practice  that  the  media  has  referred  to  as 
“paid  patriotism”  as  these  ceremonies  appeared  to  voluntarily  honor 
military  service  but  were  paid  for  by  the  military  as  part  of  an  advertising 
contract. 

In  our  2003  review  of  DOD’s  advertising  activities,  we  found  that  DOD  did 
not  have  clear  program  objectives  and  adequate  outcome  measures  in 
place  to  evaluate  the  effectiveness  of  advertising  as  part  of  its  overall 
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recruiting  effort.1  We  therefore  recommended  that  DOD  set  such 
objectives  and  develop  measures  to  monitor  advertising’s  performance 
and  make  choices  about  advertising  funding  as  part  of  the  overall 
recruiting  investment.  DOD  implemented  this  recommendation  and 
sponsored  research  intended  to  determine  how  best  to  evaluate  the 
effectiveness  of  military  advertising. 

Senate  Report  1 14-49  accompanying  a  bill  for  the  National  Defense 
Authorization  Act  for  Fiscal  Year  2016  included  a  provision  that  we 
assess  DOD’s  sponsorships,  sports  marketing,  and  advertising  activities 
for  the  active  duty,  reserve,  and  guard  military  service  components.2  This 
report  examines  the  extent  to  which  DOD  (1)  has  coordinated  its 
advertising  activities  among  the  military  service  components,3  (2)  has 
followed  commercial  best  practices  to  assess  the  effectiveness  of 
advertising  activities,  and  (3)  has  oversight  of  the  components’ 
advertising  activities. 

To  determine  the  extent  to  which  DOD  has  coordinated  its  advertising 
activities  among  the  military  service  components,  we  reviewed 
department-  and  service-level  guidance  pertaining  to  roles  and 
responsibilities  for  advertising.  We  interviewed  officials  from  the  Office  of 
the  Undersecretary  of  Defense  for  Personnel  and  Readiness,  the  Joint 
Advertising  and  Marketing  Research  Studies  (JAMRS)  office,  and  each  of 
the  military  service  components’  advertising  programs  to  discuss  any 
coordination  of  advertising  activities  that  occurs.  We  compared  the 
guidance  and  information  obtained  from  these  interviews  against  a  best 
practice  identified  by  private  sector  advertising  experts,  which  calls  for  an 
effective  use  of  coordination  in  advertising  to  improve  efficiencies,  and 
our  duplication,  overlap,  and  fragmentation  evaluation  and  management 
guide.4  This  guide  describes  how  to  identify  and  evaluate  instances  of 
fragmentation  (more  than  one  agency  involved  in  the  same  broad  area), 


^AO,  Military  Recruiting:  DOD  Needs  to  Establish  Objectives  and  Measures  to  Better 
Evaluate  Advertising’s  Effectiveness,  GAO-03-1005  (Washington,  D.C.:  Sept.  19,  2003). 

2S.  Rep.  No.  114-49,  at  77-78  (May  19,  2015),  accompanying  S.  1376. 

3The  military  service  components  included  in  this  report  are:  Army  active  duty,  reserve, 
and  Army  National  Guard;  Navy  active  duty  and  reserve;  Marine  Corps  active  duty  and 
reserve;  Air  Force  active  duty,  reserve,  and  Air  National  Guard. 

4GAO,  Fragmentation,  Overlap,  and  Duplication:  An  Evaluation  and  Management  Guide, 
GAO-1 5-49SP  (Washington,  D.C.:  Apr.  14,  2015). 
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overlap  (multiple  agencies  or  programs  with  similar  goals,  activities,  or 
beneficiaries),  and  duplication  (two  or  more  agencies  or  programs 
engaged  in  the  same  activities  or  services  to  the  same  beneficiaries) 
among  programs. 

To  determine  the  extent  to  which  DOD  has  followed  commercial  best 
practices  to  assess  the  effectiveness  of  advertising  activities,  we  took 
several  steps.  First,  we  interviewed  officials  from  the  Office  of  the  Under 
Secretary  of  Defense  for  Personnel  and  Readiness  and  each  of  the 
military  service  components  responsible  for  advertising  to  determine  what 
types  of  advertising  they  conduct  and  how  its  effectiveness  is  determined. 
Second,  to  identify  the  best  practices  for  assessing  the  effectiveness  of 
advertising  in  the  commercial  sector,  we  conducted  a  preliminary  search, 
including  a  literature  review,  and  determined  that  a  well-defined  and 
widely  accepted  list  of  best  practices  had  not  been  established. 

Therefore,  we  selected  a  nongeneralizable  sample  of  advertising 
companies  and  professional  organizations  using  a  “snowball  sampling 
methodology,”  which  consisted  of  interviewing  advertising  industry 
experts  from  an  initial  set  of  organizations  and  requesting  those  experts 
to  refer  additional  contacts  whom  we  then  interviewed.  During  these 
interviews,  we  asked  each  expert  or  organization  representative  to 
identify  best  practices  for  assessing  the  effectiveness  of  advertising.  We 
then  compiled  these  practices  into  a  list  and  provided  the  list  to  the 
organizations  for  review  and  comment.  Although  the  perspectives  of  the 
organizations  included  in  our  sample  are  not  generalizable  to  the 
advertising  industry  as  a  whole,  we  found  a  sufficient  degree  of 
consensus  among  the  sample  for  the  purpose  of  developing  criteria  for 
this  review.  Third,  we  reviewed  the  most  recent  version  of  each  military 
service  component’s  annual  marketing  and  advertising  plan,  which 
ranged  from  fiscal  year  2014  through  fiscal  year  2016,  depending  on  the 
service  component.  We  compared  each  plan  and  associated  processes 
with  the  best  practices  we  identified  for  assessing  the  effectiveness  of 
advertising  activities  and  the  Standards  for  Internal  Control  in  the  Federal 
Government,  which  calls  for  an  organization  to  have  relevant,  reliable, 
and  timely  information  and  communication  of  that  information  throughout 
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the  agency  in  order  to  achieve  all  of  its  objectives.5  Lastly,  we  interviewed 
officials  from  the  headquarters  of  each  of  the  military  service  components’ 
advertising  programs  and  representatives  from  their  contracted 
advertising  agencies,  and  reviewed  each  component’s  marketing  and 
advertising  plan  to  determine  the  extent  to  which  the  services  followed  the 
identified  best  practices. 

To  determine  the  extent  to  which  DOD  has  oversight  of  the  components’ 
advertising  activities,  we  reviewed  federal  internal  control  standards, 
which  calls  for  planning,  directing,  and  controlling  activities  to  ensure  that 
goals  and  objectives  can  be  achieved.  Further,  these  internal  control 
standards  call  for  an  agency’s  activities  to  be  directed  through  policies.6 
We  compared  these  standards  with  (1)  any  department-level  guidance  for 
advertising  or  recruiting  and  (2)  advertising  activities  and  service-level 
guidance  on  the  review  and  approval  processes  related  to  the  services’ 
and  components’  advertising  programs.  In  addition,  to  determine  how  the 
components  provide  direction  to  and  oversight  of  advertising,  we 
interviewed  officials  from  the  headquarters  of  each  of  the  military  services 
components’  advertising  programs  and  a  judgmental,  nonprobability 
sample  of  two  lower  level  recruiting  units  from  each  military  service 
component  with  advertising  responsibilities  selected  to  obtain  varying 
perspectives.  Specifically,  we  met  with  units  responsible  for  recruiting  and 
advertising  that  represent  different  regions  or  areas  of  the  United  States, 
such  as  a  recruiting  district,  squadron,  or  brigade  depending  on  the 
service.  The  subordinate  level  units  were  selected  to  obtain  distinct  views 


5See  GAO,  Standards  for  Internal  Control  in  the  Federal  Government, 
GAO/AIMD-OO-21.3.1  (Washington,  D.C.:  November  1999).  These  standards  were  in 
effect  prior  to  fiscal  year  2016  and  cover  the  time  period  of  most  of  the  service 
components’  advertising  plans.  The  standards  were  subsequently  updated.  The  updated 
standards  went  into  effect  on  October  1, 2015,  and,  consistent  with  the  old  version  of  the 
standards,  emphasize  the  need  for  relevant,  reliable,  and  timely  information  and 
communication  of  that  information  throughout  the  agency  in  order  to  achieve  all  of  its 
objectives.  See  GAO,  Standards  for  Internal  Control  in  the  Federal  Government, 

GAO-1 4-704G  (Washington,  D.C.:  Sept.  10,  2014). 

6See  GAO/AIMD-OO-21.3.1  (Washington,  D.C.:  November  1999).  These  standards  were 
in  effect  prior  to  fiscal  year  2016  and  cover  the  time  period  of  most  of  the  service 
components’  guidance  regarding  advertising.  The  standards  were  subsequently  updated. 
The  updated  standards  went  into  effect  on  October  1, 2015,  and,  consistent  with  the  old 
version  of  the  standards,  emphasize  agency  planning,  directing,  and  controlling  activities 
to  ensure  goals  and  objectives  can  be  achieved.  Further,  the  new  version  of  the  internal 
control  standards  requires  that  an  agency’s  activities  be  directed  through  policies.  See 
GAO-1 4-704G  (Washington,  D.C.:  Sept.  10,  2014). 
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within  each  component,  based  on  variations  in  the  size  of  budgets, 
recruiting  goals,  or  the  propensity  of  youth  to  serve  in  the  military  within 
the  geographical  area  of  the  unit.  Lastly,  we  reviewed  contracting  data 
from  fiscal  years  2014  through  2015  to  identify  examples  of  variations  in 
the  types  of  activities  included  in  military  service  component  advertising 
contracts.  Through  our  discussions  with  relevant  experts  and  our  review 
of  past  work,  we  determined  the  data  to  be  sufficiently  reliable  for  the 
purpose  of  corroborating  the  types  of  activities  included  in  advertising 
contracts. 

We  conducted  this  performance  audit  from  June  2015  to  May  2016  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives.  We  provide  additional 
information  about  our  scope  and  methodology  in  appendix  I. 


Background 


Under  Title  10  of  the  United  States  Code,  the  Secretary  of  each  Military 
Department  (the  Army,  the  Navy,  and  the  Air  Force)  has  the  responsibility 
to  recruit  personnel,  subject  to  the  authority,  direction,  and  control  of  the 
Secretary  of  Defense.7  As  such,  each  Secretary  has  the  authority  to 
organize  and  delegate  responsibility  for  advertising  efforts  within  each 
military  service  or  components  within  each  military  service  and,  as  a 
result,  the  organizational  structure  of  advertising  programs  and 
associated  recruiting  organizational  structure  for  each  of  the  military 
departments  differs. 

While  advertising  is  carried  out  by  the  military  services,  some  roles  and 
responsibilities  for  advertising  reside  with  the  Office  of  the  Under 
Secretary  of  Defense  for  Personnel  and  Readiness.  Within  that  office,  the 
Accessions  Policy  office  within  Manpower  and  Reserve  Affairs  has 
responsibility  for  the  (1 )  policy,  planning,  and  program  review  of  active 


Roles  and  Responsibilities 
for  DOD  Advertising 
Programs 


710  U.S.C.  §§  3013,  5013,  and  8013. 
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and  reserve  personnel,  procurement,  and  processing  and  (2) 
development,  review,  and  analysis  of  policies,  resource  levels,  and 
budgets  for  recruiting  enlisted  personnel  and  for  officer  commissioning 
programs.  One  function  for  recruiting  enlisted  personnel  and  officer 
commissioning  programs  is  advertising.  Further,  JAMRS  reports  to  the 
Director  of  DOD’s  Defense  Manpower  and  Data  Center  and  is 
responsible  for  joint  marketing  communications  and  market  research  and 
studies.  JAMRS  is  to  conduct  research  about  the  perceptions,  beliefs, 
and  attitudes  of  American  youth  as  they  relate  to  joining  the  military, 
which  is  often  referred  to  as  the  propensity  to  serve.  JAMRS  officials 
stated  that  understanding  these  factors  helps  ensure  recruiting  efforts  are 
directed  in  the  most  efficient  and  beneficial  manner.  JAMRS  has  in  the 
past  carried  out  joint  advertising  aimed  at  “influences,”  or  the  adults  that 
can  influence  or  impact  a  potential  recruit’s  decisions  regarding  military 
service.  JAMRS  also  maintains  a  database  that  is  used  across  the 
military  services  to  begin  their  outreach  to  potential  recruits. 


Types  of  Advertising 
Conducted  by  the  Military 
Service  Components 


The  military  service  components  conduct  advertising  in  support  of  their 
recruitment  missions.  Consistent  with  the  private  sector,  the  components’ 
advertising  programs  follow  a  strategy  that  considers  the  phases  of  an 
individual’s  decision-making  process,  sometimes  referred  to  as  the 
consumer  journey.  The  decision  to  enlist  in  the  military  is  a  significant 
commitment  and  can  be  affected  by  numerous  factors,  such  as  other 
employment  or  educational  opportunities  available  to  an  individual  who  is 
considering  a  career  in  the  military.  According  to  military  service  officials, 
these  phases  are  generally  characterized  by  the  components  as 
awareness,  engagement,  and  lead  generation,  as  illustrated  in  figure  1. 
The  goal  of  military  advertising  is  to  move  a  potential  recruit  through  each 
phase  and,  ultimately,  to  a  decision  to  enlist.  Further,  each  military 
service  conducts  advertising  throughout  each  phase  that  according  to 
military  service  officials  is  intended  to  communicate  and  reinforce  a 
certain  brand  or  image  among  potential  recruits,  leading  potential  recruits 
to  determine  that  a  particular  military  service  is  the  best  fit  for  their 
individual  interests,  beliefs,  or  goals. 
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Figure  1 :  Military  Service  Components’  Advertising  Phases  and  Tactics  in  Support 
of  Recruitment  Goals 
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Source:  GAO  analysis  of  Department  of  Defense  and  private  sector  advertising  information.  |  GAO-16-396 


•  Awareness.  The  military  service  components  conduct  general 
awareness  advertising  to  inform  members  of  an  audience  about  the 
opportunity  to  serve  in  the  military  and  the  distinct  characteristics  of 
each  military  service.  The  components  typically  pursue  awareness 
through  traditional  advertising  formats  such  as  television  commercials, 
print  advertisements,  and  banners  at  events  or  signs  within  a 
community. 

•  Engagement.  Advertising  focused  on  building  engagement  targets 
individuals  who  are  aware  of  the  military  as  a  career  option  and  have 
begun  to  consider  the  possibility  of  enlisting.  During  this  phase,  the 
components  seek  to  provide  recruits  with  additional  information  to  aid 
in  their  decision-making  process.  Often  this  phase  of  advertising  takes 
place  in  the  digital  environment,  as  components  seek  to  provide 
informative  social  media  posts  and  use  banner  advertisements  to 
attract  individuals  to  visit  their  websites  for  more  information.  Figure  2 
describes  a  variety  of  digital  advertising  activities  conducted  by  the 
components. 
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Figure  2:  Examples  of  Military  Service  Components’  Digital  Advertising  Activities 


Websites 

The  military  service  components’  websites  are  the  virtual  hubs  of  their  digital 
advertising  efforts.  Websites  provide  potential  recruits  with  additional 
information  such  as  qualifications  for  military  service,  types  of  positions 
available,  and  what  it  is  like  to  serve  in  the  military. 


CLICK  HERE! 


Banner  advertisements 

Banner  advertisements  are  intended  to  drive  prospective  recruits  to  the  military 
service  components’  websites.  The  components  may  purchase  banner 
advertisement  space  on  related  web  pages,  such  as  job  search  pages,  and 
users  who  click  on  the  advertisement  are  directed  to  the  components’  web 
pages,  where  they  can  learn  more  about  opportunities  to  enlist. 


Paid  search 

Paid  search  allows  advertisers  to  pay  for  their  website  to  appear  at  the  top 
of  search  results  for  specific  key  words  or  phrases.  For  example,  in  order  to 
target  health  professionals,  a  military  service  component  may  pay  a  search 
engine  for  their  website  to  prominently  appear  in  the  search  results  for  the 
query  “medical  jobs.” 


Geofencing 

The  military  service  components  can  use  geofencing,  or  setting  virtual 
barriers,  to  target  digital  advertising  to  individuals  in  a  certain  geographic 
area.  This  can  allow  the  components  to  focus  their  advertising  resources  on 
potential  recruits  near  military  installations  or  in  other  prioritized  areas. 


Data  mining 

Data  mining  refers  to  the  use  of  computer  software  to  analyze  large  data 
sets  to  identify  patterns  or  relationships  between  variables.  The  military 
service  components  can  use  this  method  to  analyze  the  large  number  of 
resumes  available  on  certain  employment  sites  to  identify  what  types  of 
candidates  may  be  most  eligible  for  and  interested  in  military  service. 


Social  media 

The  military  service  components  engage  in  a  variety  of  social  media, 
including  Facebook,  Twitter,  Linkedin,  and  others.  Social  media  provide  the 
components  the  opportunity  to  both  interact  directly  with  potential  recruits 
and  direct  them  to  their  websites  for  more  information. 


Source:  GAO  analysis  of  Department  of  Defense  and  private  sector  advertising  information.  |  GAO-16-396 
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•  Lead  Generation.  Lead  generation  advertising  targets  individuals 
who  have  considered  military  service  and  are  ready  to  discuss  the 
possibility  of  enlistment.  As  such,  lead  generation  activities  seek  to 
encourage  these  individuals  to  provide  their  contact  information  in 
order  to  schedule  an  opportunity  to  meet  with  a  recruiter.  Lead 
generation  is  often  conducted  in  person,  such  as  through  recruiters’ 
presence  at  events  like  career  fairs  or  sports  games.  Lead  generation 
may  also  be  conducted  through  other  means — such  as  direct  mail  and 
online  or  print  classified  advertisements — as  long  as  the 
advertisement  features  a  “call  to  action”  intended  to  prompt  viewers  to 
provide  their  contact  information.  Further,  the  military  services  often 
employ  the  use  of  “mobile  assets,”  such  as  large  trucks  and  trailers 
fitted  with  equipment  and  activities  intended  to  draw  crowds  and 
encourage  and  facilitate  public  interaction  with  a  recruiter  at  an  event 
in  order  to  generate  leads. 

Figure  3  shows  some  examples  of  various  types  of  military  service 
advertising  used  for  recruiting  purposes,  such  as  mobile  assets  used  at 
recruiting  events  to  advertise  a  specific  military  service,  digital  advertising 
on  social  media,  and  print  brochures. 
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Figure  3:  Examples  of  Military  Service  Advertising:  with  Equipment,  in  Social  Media, 
and  in  Print 


Source:  GAO.  I  GAO-16-396 
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DOD  Has  Taken  Steps  to 
Coordinate  Some 
Advertising  Activities 


DOD  has  taken  steps  to  coordinate  some  advertising  activities  among  the 
military  service  components.  Within  the  military  departments,  there  are 
seven  military  service  component  advertising  programs  (see  app.  II  for 
more  details)  that  compete  to  attract  recruits  from  a  relatively  small  pool 
of  individuals  that  are  eligible  for  military  service.  Private  sector 
advertising  industry  experts  we  spoke  to  emphasized  the  importance  of 
maintaining  a  unique  brand  and  strategy  for  each  of  the  service 
components  when  there  is  competition  for  a  target  audience.  According  to 
DOD  officials,  each  component  works  to  develop  a  unique  brand  that 
differentiates  the  military  services  in  order  to  compete  for  potential 
recruits.  While  industry  experts  stated  that  competition  is  inherent  to 
advertising,  these  experts  also  stated  that  coordination  can  sometimes  be 
beneficial  to  increase  efficiencies  and  effectiveness,  and  that  DOD  could 
pursue  greater  coordination  in  some  instances  to  help  address  any 
inefficiencies. 


Military  Service  Component  Advertising 

Programs  within  the  Military  Departments 

The  Department  of  the  Army 

•  Army  Active  Duty  and  Reserve 
Components  Advertising  Program 

•  Army  National  Guard  Advertising  Program 

The  Department  of  the  Navy 

•  Navy  Active  Duty  and  Reserve 
Components  Advertising  Program 

•  Marine  Corps  Active  Duty  and  Reserve 
Components  Advertising  Program 

The  Department  of  the  Air  Force 

•  Air  Force  Active  Duty  Component 
Advertising  Program 

•  Air  Force  Reserve  Component 
Advertising  Program 

•  Air  National  Guard  Component 
Advertising  Program 

Source:  GAO  analysis  of  DOD  information.  |  GAO-16-396 


Despite  the  competition  that  exists  among  the  military  components’ 
advertising  programs,  DOD  has  coordinated  certain  advertising  activities. 
For  example,  DOD  established  the  JAMRS  office  in  2002  to  create  a 
centralized  program  for  joint  market  research  and  communication. 

JAMRS  provides  the  military  components  with  information  that  informs 
their  advertising  programs  from  surveys  of  U.S.  youth  attitudes  toward 
joining  the  military,  which  change  and  evolve  considerably  over  time. 
Officials  across  the  military  components  stated  that  they  heavily  relied  on 
the  market  research  conducted  by  JAMRS  and  reported  using  the 
information  to  tailor  their  advertising  and  recruiting  activities  to  address 
the  interests  of  U.S.  youth. 

JAMRS  used  to  provide  joint  advertising  campaigns  directed  at  adults 
who  might  influence  an  individual’s  decision  to  join  the  military,  such  as 
parents  or  coaches,  known  as  influencers.  According  to  JAMRS  officials, 
joint  advertising  campaigns  of  any  type  have  not  been  carried  out  in 
recent  years  due  to  budget  constraints.  Military  service  component 
officials  stated  that  JAMRS’  joint  advertising  campaigns  had  been 
important  for  building  awareness  among  influencers  and  assisting  in 
promoting  a  positive  image  of  military  service  to  the  U.S.  public.  Further, 
these  officials  stated  that  joint  advertising  for  influencers  provided  by 
JAMRS  was  beneficial  for  all  the  military  services  and  that  due  to  the 
resources  needed  to  conduct  service-specific  advertising,  it  is  not  feasible 
for  their  components  to  conduct  additional  advertising  focused  on 
influencers. 
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Additionally,  the  Office  for  Accessions  Policy  within  the  Under  Secretary 
of  Defense  for  Personnel  and  Readiness  has  taken  specific  steps  to 
coordinate  among  the  components  to  attempt  to  increase  the 
effectiveness  of  DOD’s  advertising  or  to  address  shared  challenges  in 
advertising  that  are  crosscutting  in  nature  and  can  affect  all  of  the  military 
service  components.  For  example,  the  Office  coordinated  a  response  to 
media  and  congressional  interest  in  recent  activities  conducted  as  part  of 
sports  advertising  that  were  questioned  as  being  inappropriate.  Further, 
the  Under  Secretary  responded  to  this  crosscutting  challenge  in 
September  2015  by  issuing  interim  guidance  for  sports  marketing  that 
applied  to  all  service  components  to  prevent  inappropriate  activities  from 
occurring  as  part  of  sports  advertising.  In  addition,  DOD  officials  stated 
that,  in  an  effort  to  improve  the  effectiveness  of  DOD’s  advertising,  in 
2008  the  Office  for  Accessions  Policy  convened  a  cross-service  working 
group  of  military  service  component  advertising  officials  to  develop  a  set 
of  consistent  performance  measures  that  each  military  service  component 
could  use  to  assess  performance.  However,  DOD  officials  stated  that 
consensus  was  not  reached,  and  thus  the  measures  were  not  developed. 

Officials  from  the  military  service  components  stated  that  they  meet  about 
quarterly  with  their  counterparts  at  meetings  held  by  JAMRS  to  obtain 
their  respective  results  from  joint  market  research  conducted  by  JAMRS, 
and  at  these  meetings  they  will  on  occasion  discuss  crosscutting  issues 
that  affect  the  services’  advertising  programs.  Military  service  component 
advertising  officials  stated  that  these  discussions  have  at  times  provided 
an  opportunity  to  engage  in  discussions  on  effective  advertising  practices 
and  lessons  learned  across  service  programs.  Further,  military  service 
officials  stated  that  they  have  long-standing  working  relationships  with 
their  counterparts  among  the  services  and  that  they  do  on  occasion  share 
some  information  during  these  discussions.  However,  the  officials  stated 
that  they  may  not  share  comprehensive  details  of  best  practices  or 
lessons  learned  because  of  the  competition  for  recruits  among  the 
components  and  because  they  are  not  required  to  share  information  or 
coordinate. 
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DOD  Does  Not  Have  a 
Formal  Process  for 
Coordination  among  the 
Services,  Which  Can 
Result  in  Possible 
Unnecessary  Duplication, 
Overlap,  and 
Fragmentation 


The  private  sector  executives  and  professional  association 
representatives  we  spoke  with  stated  that  while  competition  is  an  inherent 
aspect  of  advertising,  increasing  coordination  should  be  considered  in 
order  to  reduce  inefficiencies  and  leverage  resources  effectively.  Further, 
we  have  found  in  prior  work  that  mechanisms  to  coordinate  programs  that 
address  crosscutting  issues  may  reduce  the  instances  of  potentially 
duplicative,  overlapping,  and  fragmented  efforts.8  While  DOD  has  taken 
some  steps  to  coordinate,  the  department  has  not  established  a  formal 
process  to  ensure  that  service  component  advertising  officials  have  a 
forum  to  share  information  and  address  crosscutting  issues 
systematically,  and  that  these  discussions  are  consistently  occurring,  to 
help  ensure  an  efficient  use  of  resources  in  the  competitive  environment 
for  recruitment. 


Definitions  of  Duplication,  Overlap,  and 
Fragmentation 

Duplication.  Occurs  when  two  or  more 
agencies  or  programs  are  engaged  in  the 
same  activities  or  provide  the  same  services 
to  the  same  beneficiaries. 

Overlap.  Occurs  when  multiple  agencies  or 
programs  have  similar  goals,  engage  in 
similar  activities  or  strategies  to  achieve  them, 
or  target  similar  beneficiaries. 

Fragmentation.  Refers  to  those 
circumstances  in  which  more  than  one  federal 
agency  (or  more  than  one  organization  within 
an  agency)  is  involved  in  the  same  broad  area 
of  national  need  and  opportunities  exist  to 
improve  service  delivery 

Source:  GAO.  |  GAO-16-396 


In  the  absence  of  a  formal  process  for  coordination,  we  found  examples 
of  possible  unnecessary  duplication,  overlap,  and  fragmentation  within 
DOD’s  advertising  activities.9 

•  Risk  of  Increased  Cost  of  Advertising  Media  Purchases.  Industry 
experts  and  various  military  service  officials  stated  that  as  each 
military  service  component  purchases  advertising  space  in  the  same 
media  market,  such  as  the  time  to  air  an  advertisement  on  radio  or 
television,  competition  among  the  services  could  increase  the  prices 
of  these  purchases.  Further,  they  stated  that  working  together  could 
increase  the  buying  power  for  certain  activities.  As  such,  from  a 
department-wide  perspective,  DOD  is  at  risk  of  making  duplicative 
purchases  in  the  same  market  with  no  coordination  regarding  the  cost 
and  effectiveness  of  those  purchases.  Officials  from  one  military 
service  component  acknowledged  that  the  competition  for  media 
purchases  most  likely  results  in  a  more  expensive  manner  of  doing 
business  for  the  entire  military. 


•  Multiple  Contracts  for  Similar  Functions.  While  each  service 

component’s  advertising  agency  works  with  the  component  to  develop 
a  unique  brand  that  differentiates  the  military  services,  the 


8GAO-15-49SP. 

9See  GAO,  2015  Annual  Report:  Additional  Opportunities  to  Reduce  Fragmentation, 
Overlap,  and  Duplication  and  Achieve  Other  Financial  Benefits,  GAO-1 5-404SP 
(Washington,  D.C.:  Apr.  14,  2015)  for  an  explanation  of  how  addressing  duplication, 
overlap,  and  fragmentation  can  increase  efficiencies  and  improve  effectiveness. 
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components  each  contract  for  some  functions  that  are  not  brand 
specific.  For  example,  service  officials  reported  that  each  component 
has  advertising  contracts  that  include  services  such  as  call  centers 
and  website  chat-function  support  that  respond  to  requests  for 
information  generated  by  advertising,  and  that  screen  potential 
recruits  against  general  criteria  for  military  service.  As  such,  there  are 
at  least  seven  call  center  functions  that  the  department  uses  to  screen 
potential  recruits.  Most  of  these  call  centers  are  staffed  by  contractors 
from  the  general  public  and  do  not  provide  a  service-specific  function. 
Thus  the  department  is  paying  several  different  companies  to  provide 
a  similar  service.  Advertising  industry  experts  pointed  to  functions  that 
are  not  brand  specific  as  possible  opportunities  for  obtaining 
efficiencies  through  coordination  or  consolidation.  However,  military 
service  officials  disagreed  with  this  assessment  and  stated  that 
benefits  that  might  result  from  consolidation  of  these  types  of 
functions  are  unclear  and  that  call  centers  may  provide  brand-specific 
information.  For  example,  officials  from  the  Army  National  Guard 
stated  that  trained  individuals,  sometimes  Army  National  Guard 
members,  can  offer  state-  or  territory  guard-  specific  information  when 
responding  to  calls  from  their  contracted  call  center. 

As  discussed  below,  we  also  found  examples  where  better  coordination 

within  and  among  the  military  service  components  could  increase 

efficiencies  or  effectiveness  by  addressing  the  fragmentation  of  these 

advertising  programs. 

•  Three  Advertising  Programs  within  the  Air  Force.  The  Air  Force 
has  three  components  that  each  contract  with  a  different  advertising 
agency  to  develop  and  implement  three  separate  advertising 
programs.  Maintaining  three  separate  programs  can  lead  to 
inefficiencies.  For  example,  officials  from  the  Air  Force  reserve  and 
guard  components  stated  that  they  do  not  have  the  resources  to  do 
marketing  mix  modeling.10  Marketing  mix  modeling  is  a  best  practice 
employed  by  agencies  to  determine  the  most  efficient  and  effective 
allocation  of  a  client’s  budget  toward  media  buys  including  print, 
television,  and  digital  advertising.  However,  if  the  Air  Force 
components  coordinated  on  similar  advertising  functions  that  are  not 
part  of  their  unique  branding,  such  as  marketing  mix  modeling,  they 


10See  appendix  III  for  trends  in  budgetary  data  for  each  military  service  component’s 
advertising  program. 
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could  potentially  afford  to  jointly  contract  for  such  functions.  In 
response  to  our  questions  about  why  they  had  not  coordinated  these 
programs,  officials  from  the  components  stated  that  both  the  reserve 
component  and  Air  National  Guard  need  to  focus  on  the  geographic 
location  of  potential  recruits  to  fill  vacancies,  whereas  this  is  not  a 
concern  for  the  active  duty  components.  However,  they  further  stated 
that  there  had  been  discussion  of  consolidating  some  components  in 
the  past,  possibly  the  guard  and  reserve  components,  but  that  this 
was  not  pursued.  Officials  could  not  provide  any  further  rationale  for 
requiring  separate  programs  or  for  why  further  efforts  for  consolidation 
were  not  pursued. 

•  Two  Army  Advertising  Programs.  There  are  two  Army  advertising 
programs,  one  for  the  Army  active  duty  and  reserve  components  and 
another  for  the  Army  National  Guard.  Better  coordination  between 
these  two  programs  could  result  in  more  effective  use  of  resources  for 
a  common  purpose.  Army  National  Guard  officials  stated  that  they  do 
not  have  the  resources  to  fund  some  needed  services  that  the  active 
Army  could  support.  For  example,  Army  National  Guard  officials 
stated  that  they  do  not  procure  warehouse  space  for  the  storage  of 
advertising  materials  for  guard  units  throughout  the  country  or  mobile 
assets  to  deploy  for  National  Guard  campaigns,  which  the  active  Army 
could  potentially  support.  Army  National  Guard  officials  stated  regular 
coordination  does  not  exist  between  these  two  programs  and  to 
address  these  shortcomings  they  have  begun  to  engage  in 
discussions  with  the  active  Army  component  to  further  explore 
coordination  for  these  activities.  A  January  2016  report  from  the 
National  Commission  on  the  Future  of  the  Army  stated  that  separation 
of  the  Army’s  recruiting  programs  and  associated  advertising  is 
inefficient  and  unproductive.  The  report  recommended  the 
establishment  of  pilot  programs  that  align  the  recruiting  efforts  of  the 
active  duty  Army,  Army  National  Guard,  and  Army  Reserve  as 
consolidating  the  administration  and  budgeting  of  recruiting, 
advertising,  and  branding  for  all  components  will  yield  increased 
effectiveness  and  efficiency.11 

•  Army  National  and  Local  Advertising.  We  also  found  fragmentation 
in  the  Army’s  active  and  reserve  advertising  programs  resulting  in 
coordination  and  communication  challenges.  Within  the  active  and 


^National  Commission  on  the  Future  of  the  Army,  Report  to  the  President  and  the 
Congress  of  the  United  States,  (January  28,  2016). 
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reserve  Army  components’  advertising  program,  the  Army  has  two 
organizations  that  share  some  degree  of  responsibility  for  carrying  out 
advertising  activities:  its  Army  Marketing  and  Research  Group 
advertising  office  which  directs  the  Army’s  national  level  advertising 
and  U.S.  Army  Recruiting  Command  which  directs  local  advertising 
for  recruiting.  Senior  officials  from  both  organizations  stated  their 
organization  is  responsible  for  lead  generation,  and  recruiters 
generate  leads  at  both  national  and  local  events. 

While  we  acknowledge  the  importance  of  maintaining  a  unique  brand  for 
the  service  components,  a  formal  process  for  coordination  could  allow  the 
military  service  components  to  more  effectively  share  best  practices.  As 
part  of  this  process,  a  review  of  DOD’s  existing  programs  could  potentially 
identify  opportunities  to  obtain  efficiencies  by  reducing  unnecessary 
duplication,  overlap,  and  fragmentation  that  may  exist  within  and  among 
military  service  components’  advertising  programs.  As  numerous 
department  officials  cited  a  lack  of  needed  resources  to  appropriately 
carry  out  advertising  in  order  to  best  execute  the  difficult  task  of  recruiting, 
identifying  and  reducing  unnecessary  duplication,  overlap,  and 
fragmentation  could  potentially  free  up  additional  resources.  In  the 
absence  of  a  formal  process  for  coordination  of  the  department’s  several 
advertising  programs,  DOD  may  not  be  positioned  to  best  leverage  its 
advertising  funds. 
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DOD  Has  Generally 
Followed  Commercial 
Best  Practices  for 
Assessing  the 
Effectiveness  of 
Advertising,  but 
Components  Vary  in 
Their  Ability  to 
Determine  Whether 
Their  Activities  Are 
Generating 
Recruitment  Leads 


With  some  exceptions,  DOD — through  its  military  service  components 
and  together  with  their  contracted  advertising  agencies — generally  follows 
commercial  best  practices  that  we  identified  for  assessing  the 
effectiveness  of  advertising,  shown  in  table  1 .  We  determined  that  a  well- 
defined  and  widely  accepted  list  of  best  practices  had  not  been 
established;  therefore,  we  asked  a  nongeneralizable  sample  of 
advertising  experts  and  professional  associations  to  identify  best 
practices.  We  compiled  and  condensed  the  identified  best  practices  into  a 
list  that  we  then  validated  with  those  same  industry  experts.  However, 
when  we  compared  the  military  service  components  advertising  programs 
against  the  best  practices  that  we  identified,  we  found  variations  among 
the  components  in  establishing  measurable  goals.  Further,  differences 
across  the  components  in  their  processes  for  collecting  and  reviewing 
performance  data  have  resulted  in  varying  abilities  to  measure  the 
effectiveness  of  advertising  on  generating  leads,  especially  at  the  local 
level. 


Table  1:  Commercial  Best  Practices  That  We  Identified  for  Assessing  the  Effectiveness  of  Advertising 


Organizational  structure 

•  Establish  effective  roles  and  responsibilities,  especially  to  remove  bias  in  measuring 
performance 

•  Ensure  coordinator  role  is  assigned  if  multiple  vendors  are  used  to  maintain  a  cohesive 
advertising  strategy 

Planning 

•  Develop  an  evaluation  framework  that  identifies  the  target  audience  and  includes  measureable 
goals 

•  Use  sophisticated  statistical  modeling,  such  as  marketing  mix  modeling,3  to  determine 
appropriate  spending  strategy 

Measurement 

•  Conduct  ongoing  analysis  of  performance  using  industry  standard  measures  appropriate  for  the 
purpose  of  the  advertising  activity 

•  Seek  to  develop  understanding  of  how  outcomes  can  be  attributed  to  advertising 

Source:  GAO  analysis  of  private  sector  advertising  information.  |  GAO-16-396 


According  to  industry  experts,  “marketing  mix  modeling”  refers  to  the  use  of  statistical  analysis  to 
estimate  the  effects  of  various  marketing  tactics  on  desired  behavior  (such  as  enlistment)  and  project 
the  ideal  or  most  efficient  mix  of  tactics  to  result  in  that  behavior. 
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Components  Have 
Structured  Their 
Advertising  Programs’ 
Organization  to  Safeguard 
against  Bias  in 
Performance 
Measurement  and 
Coordinate  Vendors 


We  found  that  the  military  service  components  have  largely  established 
roles  and  responsibilities  to  address  the  need  for  unbiased  performance 
information,  and  ensured  that  coordinator  roles  were  assigned  when 
multiple  vendors  were  used.  Industry  experts  we  spoke  with  stated  that 
advertising  decisionmakers  should  consider  whether  to  distribute 
responsibilities  for  various  aspects  of  advertising — including  creative 
development,  media  buying,  and  performance  analysis — across  different 
vendors,  but  when  doing  so,  should  ensure  that  a  coordinator  role  is 
assigned  to  maintain  a  cohesive  advertising  strategy.  While  the  majority 
of  the  military  service  components  rely  heavily  on  their  contracted 
advertising  agencies  to  assess  the  effectiveness  of  their  advertising 
activities,  they  also  adhered  to  these  best  practices,  as  described  in  the 
examples  below.12 


Consistent  with  this  commercial  best  practice  that  we  identified,  for 

example,  we  found  the  following: 

•  A  Marine  Corps  official  responsible  for  overseeing  the  component’s 
advertising  program  stated  that  four  Marine  Corps  project  officers  are 
designated  to  work  closely  with  its  advertising  agency  in  monitoring 
and  analyzing  performance  across  key  areas. 

•  Officials  from  the  Army,  which  has  the  largest  advertising  budget  of 
the  components,  stated  that  the  Army  had  additionally  contracted  with 
third-party  research  firms  to  further  assess  the  effectiveness  of  its 
advertising. 

•  All  of  the  components  independently  review  quantitative  performance 
data,  which  they  may  access  through  their  own  data  systems  or 
reports  provided  by  the  advertising  agencies.  Military  service  officials 
stated  that  monitoring  such  data  can  help  them  to  assess  the 
performance  of  both  their  advertising  activities  and  contracted 
advertising  agencies. 

•  Each  of  the  components  also  cited  research  conducted  by  DOD’s 
JAMRS  program  as  a  source  of  unbiased  performance  data.  For 


12During  the  time  of  our  review,  the  Navy  and  the  Army  National  Guard  told  us  that  they 
did  not  have  contracts  with  advertising  agencies  in  place.  The  Navy  has  awarded  a 
contract,  but  its  execution  is  on  hold  pending  a  bid  protest  currently  before  the  Court  of 
Federal  Claims.  The  Army  National  Guard  anticipates  that  its  next  contract  with  an 
advertising  agency  will  be  effective  in  2018. 
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example,  JAMRS  conducts  a  quarterly  advertising  tracking  survey, 
evaluating  the  target  audience’s  recall  and  reactions  to  the 
components’  television  advertising  campaigns,  which  service  officials 
stated  can  help  the  components  assess  the  effectiveness  of  that 
specific  type  of  advertising. 

•  As  each  component  contracts  with  a  lead  advertising  agency,  the 
components  typically  adhere  to  this  best  practice  as  the  agency  fulfils 
the  coordinator  role  in  conjunction  with  the  component. 


Components  Vary  in  the 
Extent  to  Which  They 
Meet  Commercial  Best 
Practices  on  Planning 
That  We  Identified 


While  all  of  the  military  service  components  develop  evaluation 
frameworks  that  identify  a  target  audience  in  accordance  with  the 
commercial  best  practice,  the  goals  set  by  the  components  vary  in 
measurability.  In  our  prior  work,  we  have  also  found  that  high-performing 
organizations  have  goals  that  are  aligned  with  performance 
management.13  We  found  that  almost  all  of  the  service  components 
typically  develop  an  annual  advertising  plan,  including  information  about 
their  target  market  and  goals  to  be  met  by  advertising  that  year.14  All  of 
the  plans  we  reviewed  demonstrated  a  detailed  understanding  of  the 
target  market.  For  example,  the  Marine  Corps’  fiscal  year  2015  plan 
recognizes  the  varying  levels  of  the  propensity  to  join  the  military  among 
its  target  market  and  identifies  an  opportunity  to  focus  on  “the  movable 
middle” — the  portion  of  prospective  recruits  who  are  not  currently  inclined 
to  join  the  military  but  might  be  willing  to  consider  it.  Likewise,  reflecting 
its  organizational  structure  of  54  state  and  territorial  units,  the  Air  National 
Guard’s  2015  annual  advertising  plan  emphasizes  targeting  advertising  to 
audiences  located  near  units  where  career  opportunities  are  available. 


However,  the  commercial  best  practice  on  planning  that  we  identified  also 
calls  for  advertising  plans  to  include  measurable  goals.  In  our  review  of 
the  most  recent  version  of  each  component’s  annual  advertising  plan,  we 
found  that  the  components’  plans  varied  in  the  extent  to  which  their  goals 


13GAO,  High-Performing  Organizations:  Metrics,  Means,  and  Mechanisms  for  Achieving 
High-Performance  in  the  21st  Century  Public  Management  Environment ,  GAO-04-343SP 
(Washington,  D.C.:  February  2004). 

14The  Navy  told  us  that  its  most  recent  advertising  plan  was  completed  for  fiscal  year 
2013,  because  the  Navy  has  been  in  the  process  of  awarding  a  new  advertising  agency 
contract  since  that  time. 
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can  be  measured.15  For  example,  the  Army  plan  we  reviewed  identifies  a 
series  of  marketing  objectives  supported  by  measurable,  numeric  goals 
related  to  the  public  perception  of  the  Army.  In  contrast,  the  Marine  Corps 
and  active  duty  Air  Force  plans  state  goals  that  relate  to  emphasizing 
positive  aspects  of  the  components,  such  as  diversity  and  core  values, 
but  these  goals  are  not  numeric  and  measurable  with  related 
performance  measures.  For  example,  the  most  recent  annual  advertising 
plan  of  the  Marine  Corps  includes  a  goal  to  weave  diversity  into  all 
advertising  efforts,  while  the  Air  Force  annual  plan  contains  a  goal  that  all 
advertising  tells  the  Air  Force  a  story  in  a  way  that  highlights  Air  Force 
core  values.  Neither  plan  includes  information  on  how  these  goals  would 
be  measured.  We  also  found  that  the  annual  plans  of  the  Air  Force 
National  Guard  and  Air  Force  reserve  had  a  mixture  of  the  types  of  goals 
identified,  some  of  which  could  be  measured.  In  addition,  Army  National 
Guard  officials  stated  that  while  it  requires  each  state  unit  to  create  an 
annual  plan  that  includes  goals,  the  Army  National  Guard  headquarters 
does  not  establish  annual  goals  at  a  national  level.  Army  National  Guard 
officials  stated  that  there  is  not  a  national  level  annual  advertising  plan 
with  associated  goals  because  each  state  unit  has  a  unique  recruiting 
mission,  including  the  types  of  positions  that  need  to  be  filled  and  the 
demographic  makeup  of  the  target  audience.  The  absence  of  measurable 
goals  at  the  national  level  may  limit  Army  National  Guard  headquarters’ 
ability  to  determine  the  success  of  any  national  level  advertising  efforts  or 
to  distribute  advertising  funds  strategically  and  efficiently  among  the  state 
units.  Military  service  officials  stated  that  the  annual  advertising 
campaigns  and  associated  advertising  plans  for  each  component  can 
change  considerably  from  year  to  year.  Ensuring  that  future  iterations  of 
each  component’s  annual  advertising  plan  contain  measurable  goals 
could  enhance  their  ability  to  demonstrate  the  success  of  their  advertising 
programs. 

The  components  also  vary  in  their  use  of  sophisticated  modeling  to 
determine  how  to  distribute  available  advertising  funds  across  different 
types  of  advertising  (e.g.,  television,  print,  Internet,  etc.),  which  was  a  key 
planning  best  practice  cited  by  industry  experts.  Specifically,  industry 
experts  stated  it  is  a  best  practice  to  use  some  form  of  modeling,  such  as 
marketing  mix  modeling,  to  determine  the  optimum  distribution  of 


15As  the  most  recent  Navy  advertising  plan  was  completed  for  fiscal  year  201 3,  we  did  not 
assess  that  version  of  the  Navy’s  plan  because  officials  stated  that  it  is  out  of  date. 
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advertising  funds.  Officials  from  the  active  duty  Army,  Marine  Corps,  and 
Air  Force  active  duty  components  stated  that  they  use  such  modeling, 
provided  by  their  contracted  advertising  agencies,  to  determine  how  to 
spend  their  advertising  funds.  However,  the  remaining  components  do  not 
currently  leverage  marketing  mix  modeling,  and  Air  Force  Reserve  and 
Army  National  Guard  officials  cited  its  high  cost  as  a  barrier.  Industry 
experts  acknowledged  that,  while  marketing  mix  modeling  is  a  best 
practice,  it  can  present  a  significant  expense,  and  other  methods  of 
modeling  may  be  more  appropriate  in  some  situations.  As  such,  Army 
National  Guard  officials  stated  that  marketing  mix  modeling  might  not  be 
best  suited  for  modeling  when  mission  requirements  fluctuate  year  to  year 
or  when  geographic  location  of  vacancies  are  of  a  primary  concern. 


Components  Have  Taken 
Some  Steps  to  Measure 
Performance,  but  Some 
Have  Insufficient  Data  to 
Measure  Whether 
Outcomes  Are  Attributable 
to  Advertising 

Components  Successfully 
Applied  Industry  Standard 
Measures  for  Two  of  the  Three 
Purposes  of  Advertising 
Activities 


We  found  that  the  military  service  components  successfully  applied 
industry  standard  measures  appropriate  for  two  of  the  purposes  of 
advertising  activities — awareness  and  engagement —  but  varied  in  their 
ability  to  assess  the  effectiveness  of  advertising  related  to  the  third 
activity,  lead  generation.  For  example,  to  measure  effectiveness  of 
awareness  related  advertising,  such  as  television  commercials  or  national 
print  advertisements,  the  components  typically  measure  the  number  of 
times  an  advertisement  is  viewed  by  a  member  of  the  audience,  numbers 
known  as  impressions.  To  measure  engagement,  the  components  use  a 
range  of  real-time  digital  analytics,  including  click-through  rates16  and 
social  media  “likes,”  among  others,  when  they  conduct  digital  advertising. 
According  to  military  officials  and  representatives  of  their  advertising 
agencies,  it  is  an  industry  standard  to  directly  link  the  results 
demonstrated  by  these  analytics  to  the  purchase  of  advertising  for 
awareness  and  engagement.  The  components  rely  on  their  advertising 


16“Click-through  rates”  are  the  percentages  of  people  exposed  to  an  advertisement  who 
click  on  it  to  access  the  advertised  website. 
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agencies  to  negotiate  these  purchases  with  the  intent  of  achieving  a  level 
of  performance  that  is  consistent  with  industry  standards  for  these  forms 
of  advertising.17 

In  contrast,  we  found  that  the  components  varied  in  their  ability  to  use 
industry  standard  performance  measures  to  assess  the  effectiveness  of 
advertising  activities  focused  on  lead  generation.  Service  officials  stated 
that  such  activities  can  include  recruiter  booths  at  events,  direct  mail,  and 
certain  types  of  print  or  digital  advertising,  for  which  performance  is 
measured  on  the  basis  of  how  many  leads  are  generated  by  the  activity. 
Whereas  military  service  officials  stated  that  advertising  focused  on 
generating  awareness  and  engagement  is  generally  executed  at  the 
national  level,  lead  generation  activities  are  often  carried  out  at  the  local 
level  and  can  depend  on  recruiters’  familiarity  and  knowledge  of  their 
local  markets.  For  example,  service  officials  stated  that  recruiters  may 
select  high  school  populations  to  receive  direct  mailings  or  interact  with 
potential  recruits  at  local  events  such  as  career  fairs  or  sports  games  and 
may  also  be  responsible  for  identifying  the  number  of  leads  generated  by 
such  activities.  As  a  result,  the  responsibility  for  measuring  the 
performance  of  locally  executed  lead  generation  advertising  activities  is 
carried  out,  in  part,  at  the  local  level,  and  some  components  do  not  obtain 
or  measure  the  performance  data  needed  to  assess  the  effectiveness  of 
these  activities. 


Shortcomings  in  Measuring 
Lead  Generation  Contribute  to 
Difficulties  in  Understanding 
How  Outcomes  Can  Be 
Attributed  to  Advertising 


Insufficient  data  to  measure  the  performance  of  local  lead  generating 
activities  has  diminished  some  of  the  components’  ability  to  understand 
how  to  attribute  outcomes  to  specific  advertising  activities.  According  to 
industry  experts  and  our  prior  work,  determining  the  precise  impact  of 
advertising  on  outcomes,  such  as  recruitment,  is  inherently  challenging,  in 
part  due  to  the  concurrent  effects  of  external  factors,  such  as  the 
influence  of  family  support  and  the  availability  of  other  career  or 
educational  opportunities.18  In  addition,  the  length  and  complexity  of  a 
decision  to  enlist  in  the  military  necessitates  the  use  of  multiple  types  of 
advertising  throughout  the  recruiting  process,  and  an  individual  may 
ultimately  decide  to  enlist  months  or  years  after  first  being  exposed  to 
advertising  by  the  military.  Although  industry  experts  acknowledged  that 


17DOD  officials  stated  that  advertising  agencies  may  subcontract  this  role  to  media  buying 
companies  that  specialize  in  the  negotiation  of  the  placement  and  price  of  advertising. 

18GAO-03-1005. 
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understanding  how  outcomes  are  impacted  by  advertising  is  a  challenge 
in  both  the  private  sector  in  general  and  for  the  military  in  particular,  they 
stated  it  is  nonetheless  important  for  advertising  managers  to  develop  an 
understanding  of  how  outcomes  can  be  attributed  to  advertising. 

With  the  exception  of  certain  new  requirements  specific  to  sports 
advertising  contracts,  DOD  does  not  require  that  the  service  components 
measure  the  performance  of  their  advertising  activities,  and  the  extent  to 
which  the  components  can  currently  measure  the  effectiveness  of  their 
advertising  activities  varies.  As  such,  measurement  and  monitoring 
performance  is  at  the  discretion  of  the  military  service  components,  and 
there  is  variation  in  the  components’  processes  for  collecting  and 
reviewing  advertising  activity  performance  data.  According  to  service 
officials,  we  found  that  some  components  either  do  not  collect  local 
advertising  performance  data  related  to  lead  generation  in  their  systems 
or  cited  issues  with  the  reliability  of  such  data.  Specifically,  in  terms  of 
collecting  this  type  of  performance  data,  service  component  officials 
stated  the  following: 

•  The  Marine  Corps’  leads  tracking  system  allows  the  Marine  Corps  to 
link  multiple  exposures  to  advertising — such  as  a  direct  mailing  or 
interaction  with  a  recruiter  at  an  event — to  a  prospective  recruit’s  lead 
record,  enhancing  the  ability  to  analyze  performance  in  lead 
generation. 

•  While  the  Navy  requires  that  performance  data  for  local  advertising  be 
entered  into  its  data  systems,  currently  it  does  not  have  the  capability 
to  analyze  performance  in  the  same  manner  and  cannot  attribute 
potential  leads  to  multiple  exposures  to  advertising. 

•  Air  Force  active  duty  recruiting  squadrons  are  not  required  to  report  to 
their  headquarters  on  the  performance  of  their  advertising  activities, 
including  the  performance  of  local  advertising  activities  in  lead 
generation.  However,  Air  Force  officials  stated  they  conduct  and 
assess  the  performance  of  the  majority  of  their  advertising  at  the 
headquarters  level,  and  a  comparatively  small  portion  of  advertising 
funds  are  distributed  to  the  squadrons  for  local  advertising  activities. 

•  Both  the  Air  Force  Reserve  and  Air  National  Guard  require  that 
recruiters  submit  performance  data,  such  as  attendance  and  leads 
generated  for  any  local  advertising  activity  and  that  headquarters 
officials  review  these  data  in  their  lead  tracking  data  systems. 


Page  23 


GAO-16-396  DOD  Advertising 


•  The  Army  National  Guard  does  not  routinely  require  state  units  to 
provide  headquarters  with  performance  data  related  to  advertising, 
including  lead  generation. 

•  The  Army  requires  in  its  policy  that  subheadquarters  units  submit 
advertising  performance  data  for  headquarters  review,  but  Army 
advertising  officials  cited  concerns  with  the  reliability  of  these  data  for 
lead  generation. 

Regarding  the  Army’s  concerns  regarding  unreliable  data,  Army  officials 
stated  that  when  leads  are  collected  by  recruiters  at  advertising  events,  in 
many  cases  those  leads  are  coded  in  their  data  system  as  “recruiter 
generated,”  rather  than  being  attributed  to  the  appropriate  advertising 
activity.  As  a  result,  Army  officials  stated  that  although  they  believe  their 
locally  executed  advertising  activities  are  a  good  investment,  they  do  not 
have  sufficient  evidence  to  demonstrate  their  effectiveness.  For  example, 
while  Army  Recruiting  Command  officials  cited  “register  to  win”  giveaways 
for  promotional  items  at  local  events  as  an  effective  lead  generator,  Army 
Marketing  and  Research  Group  officials  stated  they  do  not  have  data  to 
support  the  effectiveness  of  this  advertising  activity  and  questioned  the 
quality  of  the  leads  produced.  Army  officials  stated  that  they  are  currently 
working  to  address  this  issue  to  ensure  leads  are  properly  coded  and  thus 
improve  the  reliability  of  this  data.  We  did  not  find  the  proper  coding  of 
lead  generation  to  be  an  issue  for  other  service  components;  for  example, 
according  to  service  officials,  recruiters  from  the  Marine  Corps  and  active 
Air  Force  that  attend  an  event  typically  record  a  potential  lead’s 
information  into  an  electronic  tablet  that  automatically  assigns  the  proper 
event  code  for  the  lead  based  on  the  recruiter’s  location  and  the  date  and 
time  of  the  lead’s  entry. 

Federal  internal  controls  standards  state  that  program  managers  need 
appropriate  data  to  determine  whether  they  are  meeting  stated  goals  and 
achieving  an  effective  and  efficient  use  of  resources.19  Without  access  to 


1 Q 

See  GAO,  Standards  for  Internal  Control  in  the  Federal  Government, 
GAO/AIMD-OO-21.3.1  (Washington,  D.C.:  November  1999).  These  standards  were  in 
effect  prior  to  fiscal  year  2016  and  cover  the  time  period  of  most  of  service  components’ 
advertising  plans.  The  standards  were  subsequently  updated.  The  updated  standards 
went  into  effect  on  October  1 , 201 5,  and,  consistent  with  the  old  version  of  the  standards, 
emphasize  the  need  for  relevant,  reliable,  and  timely  information  and  communication  of 
that  information  throughout  the  agency  in  order  to  achieve  all  of  its  objectives.  See  GAO, 
Standards  for  Internal  Control  in  the  Federal  Government,  GAO-14-704G  (Washington, 
D.C.:  Sept.  10,  2014). 
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the  necessary  performance  data,  such  as  reliable  leads  collection  and 
attribution  data,  a  military  service  component  may  be  limited  in  its  ability 
to  measure  the  performance  of  its  advertising  activities  against  stated 
goals.  Without  processes  in  place  to  facilitate  the  measurement  and 
monitoring  of  advertising  performance  across  all  levels — especially  at  the 
local  level — the  military  service  components  may  be  unable  to  ensure 
advertising  dollars  are  used  efficiently  to  help  meet  stated  recruiting  goals 
could  result  in  inefficient  use  of  advertising  dollars. 

In  the  Absence  of 
Policy,  DOD  Does  Not 
Have  Comprehensive 
Oversight  of  Its 
Components’ 
Advertising  Activities 

DOD  does  not  have  comprehensive  oversight  of  the  military  service 
components’  advertising  activities,  as  it  does  not  have  a  policy  that 
defines  its  oversight  role  as  well  as  procedures  to  guide  the  components’ 
respective  advertising  activities.  Federal  internal  control  standards  for  an 
agency’s  organizational  structure  call  for  planning,  directing,  and 
controlling  activities  to  ensure  goals  and  objectives  can  be  achieved. 
Further,  these  standards  require  that  an  agency’s  activities  be  directed  by 
policies  and  that  management  should  provide  appropriate  oversight  of 
activities.20  When  DOD  issues  policy,  such  as  directives  and  instructions, 
the  department  requires  that  the  policy  establishes  roles  and 
responsibilities  and  defines  the  procedures  that  are  to  be  followed  by  all 
defense  offices  and  organizations  that  are  involved  with  the  activity  or 
program  the  policy  directs. 

DOD’s  Components 
Oversee  Their  Own 
Programs,  Which  May 

Have  Led  to  Negative 
Effects  in  Some  Instances 

As  there  is  no  department-wide  policy  that  defines  DOD’s  role  in 
overseeing  advertising  activities  or  the  procedures  that  should  be 
followed  when  the  components  carry  out  their  advertising  activities,  the 
department’s  advertising  activities  are  overseen  at  the  military  service 
level  and  in  some  cases,  within  a  service’s  individual  active,  reserve,  or 
guard  components.  Each  military  service  component  has  component- 
level  policy  and  guidance  that  defines  the  overall  objectives  of  its 
advertising  program  and  sets  forth  the  roles  and  responsibilities  for  the 

20See  GAO/AIM D-00-21 .3.1  (Washington,  D.C.:  November  1999).  These  standards  were 
in  effect  prior  to  fiscal  year  2016  and  cover  the  time  period  of  most  of  the  service 
components’  guidance  regarding  advertising.  The  standards  were  subsequently  updated. 
The  updated  standards  went  into  effect  on  October  1, 2015,  and,  consistent  with  the  old 
version  of  the  standards,  emphasize  agency  planning,  directing,  and  controlling  activities 
to  ensure  goals  and  objectives  can  be  achieved.  Further,  the  new  version  of  the  internal 
control  standards  requires  that  an  agency’s  activities  be  directed  through  policies.  See 
GAO-1 4-704G  (Washington,  D.C.:  Sept.  10,  2014). 
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component’s  advertising  program.21  The  high-level  policy  is  in  many 
cases  supplemented  by  yearly  guidance  that  communicates  the  recruiting 
goals  and  the  priorities  of  the  commanding  officer  or  senior  leader 
responsible  for  a  component’s  advertising.  However,  the  high-level 
policies  issued  by  each  service  component  for  its  advertising  program 
vary  considerably  in  the  level  of  oversight  and  direction  provided,  as  the 
procedures  the  components  are  to  follow  as  they  carry  out  their 
respective  advertising  activities  have  not  been  defined  by  DOD  policy.  For 
example,  the  Navy’s  guidance  includes  financial  thresholds  for 
headquarters’  level  review  of  certain  expenditures  and  the  types  of  cost- 
effectiveness  reviews  that  must  be  performed,22  while  other  components’ 
guidance  does  not  address  the  review  of  expenditures.  The  Marine  Corps 
guidance  specifies  some  advertising  activities  that  are  prohibited, 
whereas  other  components’  guidance — including  the  Air  Force — does  not 
include  this  type  of  detail.23 

The  variation  in  the  oversight  and  direction  provided  to  each  service 
component’s  advertising  program  has  allowed  for  an  inconsistent 
understanding  of  digital  advertising  rules  and  regulations  and  for 
negatively  perceived  activities  to  occur  in  some  instances. 

•  Digital  Advertising.  We  found  differences  in  the  understanding  of 
rules  and  regulations  that  apply  to  digital  advertising  activities.  The 
government  is  restricted  in  tracking  the  digital  behavior  of  those  that 


21Department  of  the  Army  Regulation  601-208,  The  Army  Brand  and  Marketing  Program 
(July  16,  2013);  Department  of  the  Navy  Recruiting  Command  Instruction  1140.3G,  Field 
Advertising  Manual  (June  12,  2012);  Marine  Corps  Order  1 130.56D,  Total  Force 
Recruiting  (May  26,  2009);  Air  Force  Recruiting  Service  Instruction  36-2001;  Recruiting 
Procedures  for  the  Air  Force  (August  1, 2012);  Air  Force  Reserve  Command  Instruction 
36-2003,  Air  Force  Reserve  Advertising  Program  (July  1 , 2014);  Air  National  Guard 
Instruction  36-2602,  Air  National  Guard  Recruiting  and  Retention  Programs  (February  21 , 
2012);  National  Guard  Bureau  Regulation  601-1,  Army  National  Guard  Strength 
Maintenance  Program  (April  28,  2006). 

22Navy  Recruiting  Command  Instruction  1 140.3G,  Field  Advertising  Manual  (Sept.  3, 
2015). 

23Marine  Corps  Order  5720.75,  Standard  Operating  Procedures  for  Marine  Corps 
Recruiting  Command  Marketing  and  Public  Affairs  (June  4,  2012. 
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visit  government  websites.24  However,  we  observed  differing  opinions 
and  understanding  among  military  service  officials  regarding  what 
types  of  digital  tracking  was  permissible.  Officials  from  the  Air  Force 
active  duty  component’s  advertising  program  and  representatives 
from  their  contracted  advertising  agency  stated  that  persistent 
tracking  would  be  used  in  their  new  website  development  and  that 
these  plans  adhere  to  relevant  DOD  guidance.  Further,  they  stated 
that  subsequent  advertising  could  be  sent  to  individuals  after  visiting 
the  Air  Force  website.  Officials  from  the  Office  of  the  Under  Secretary 
of  Defense  for  Personnel  and  Readiness  stated  that  they  had 
reviewed  the  plans  of  the  Air  Force  regarding  digital  tracking  and 
subsequent  advertising  and  agreed  that  these  actions  would  be 
permissible  and  in  accordance  with  DOD  guidance.  However,  officials 
from  other  military  services  we  spoke  with  described  these  same 
digital  advertising  strategies  as  not  allowed  per  DOD  guidance  and 
stated  that  they  could  not  pursue  digital  tracking  similar  to  that  of  the 
private  sector.  The  other  services  could  be  at  a  disadvantage  as  a 
result  of  these  different  interpretations  of  regulations. 

•  Sports-Related  Advertising.  While  contracting  with  sports  teams 
and  events  for  advertising  is  widely  practiced  among  the  military 
service  components,  media  reports  from  2015  and  congressional 
attention  revealed  that  some  components’  contracts  with  professional 
sports  teams  included  provisions  to  conduct  ceremonies  that  honored 
servicemembers  and  provide  items  such  as  tickets  to  games,  which 
were  perceived  to  be  inappropriate  and  came  at  a  cost  to  the  federal 
government.  In  our  review  of  contracts  from  fiscal  years  2013  and 
2014,  we  found  several  contracts  with  a  marketing  firm  that  connects 
brands  to  collegiate  sports  teams  that  included  costs  for  honorary  or 
swearing  in  ceremonies  and  items  that  are  personal  in  nature.  For 
example,  we  identified  a  contract  spanning  fiscal  years  2013  and 
2014  in  which  a  major  public  university  received  $20,000  from  a  state 
Army  National  Guard  unit  for  two  swearing  in  ceremonies  of  recruits 
conducted  at  sporting  events,  and  another  state  Army  National  Guard 
unit  contracted  in  fiscal  year  2014  for  a  “VIP  experience”  at  university 
sporting  events  costing  approximately  $8,700. 


240ffice  of  Management  and  Budget  Memorandum  M-10-22,  Guidance  for  Online  use  of 
Web  Measurement  and  Customization  Technologies  (June  25,  2010)  restricts  all 
government  agencies’  tracking  of  digital  behavior  in  some  ways.  DOD  Instruction  8550.01 , 
DOD  Internet  Services  and  Internet-Based  Capabilities  (Sept.  1 1 , 2012)  establishes  DOD- 
specific  policy  regarding  digital  tracking. 
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DOD  Has  Taken  Some 
Steps  to  Improve 
Oversight  but  Does  Not 
Have  a  Department-Wide 
Policy  to  Guide  the 
Components’  Advertising 
Programs 


Subsequent  to  the  negative  media  reports  in  2015  and  resulting  DOD 
reviews  in  September  2015,  the  acting  Under  Secretary  for  Personnel 
and  Readiness  issued  interim  guidance  that  provided  more  focused 
direction  and  strengthened  oversight  of  sports  marketing  and  advertising 
contracts.  The  interim  guidance  acknowledged  the  inappropriateness  of 
paying  for  recognition  or  swearing  in  ceremonies  during  sports  events  or 
the  inclusion  of  items  that  are  personal  in  nature,  typically  sports  tickets  or 
parking,  for  which  the  receipt  of  those  items  is  not  clear  or  controlled. 
Specifically,  this  guidance  (1)  requires  that  a  senior  military  component 
reviewing  official  approve  sports  marketing  contracts,  (2)  prohibits  paying 
for  recognition  ceremonies,  (3)  restricts  items  that  are  personal  in  nature, 
and  (4)  requires  reporting  and  analysis  on  the  returns  generated  by  larger 
sports  partnerships. 

While  the  issuance  of  the  interim  DOD-level  sports  marketing  guidance  is 
a  positive  step,  a  senior  DOD  official  acknowledged  that  the  interim 
guidance  was  vague  regarding  how  some  of  these  new  requirements  are 
to  be  implemented,  did  not  include  some  monetary  thresholds  that  it  could 
have  identified,  and  applied  only  to  sports  marketing  and  advertising 
contracts.  As  such,  in  our  discussions  with  service  officials  responsible  for 
carrying  out  local  level  advertising,  we  observed  discrepancies  in  how  the 
guidance  was  being  interpreted.  For  example,  there  were  differences  in 
opinions  among  local  level  advertising  officials  about  the  inclusion  of 
honorary  or  swearing  in  ceremonies  in  contracts  with  sports  teams.  One 
local  level  official  stated  that  the  recent  interim  guidance  now  restricts  any 
mention  of  an  honorary  ceremony  in  a  contract,  while  other  officials  stated 
that  such  activities  could  still  be  included  in  a  contract  as  long  as  they 
were  listed  with  a  cost  of  $0.00  and  considered  “added  value.”  Further,  as 
the  interim  guidance  only  applies  to  sports  advertising,  it  does  not 
address  the  procedures  that  the  components  should  follow  when  carrying 
out  other  types  of  advertising.  For  example,  the  components  may  contract 
to  carry  out  advertising  at  music  concerts  or  festivals.  These  events  are 
similar  in  nature  to  sporting  events  as  they  may  also  allow  for  the 
inclusion  of  tickets  other  premium  items  in  advertising  contracts,  but 
currently  there  is  no  DOD-wide  advertising  guidance  that  prevents  the 
inclusion  of  such  items  in  contracts. 

DOD  officials  stated  that  they  are  currently  in  the  beginning  stages  of 
developing  guidance  for  advertising  that  is  more  comprehensive  to 
replace  and  build  upon  the  interim  guidance  that  was  focused  solely  on 
sports-related  advertising,  but  the  details  of  what  this  proposed  expanded 
guidance  will  cover  are  not  clear.  These  officials  stated  that  the 
department’s  process  for  development,  coordination,  approval,  and 
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publication  of  new  DOD  guidance  can  be  a  lengthy  one.  Further,  they 
stated  that  they  planned  to  review  the  implementation  of  the  interim 
guidance  and  incorporate  any  lessons  learned  from  the  issuance  of  the 
September  2015  guidance,  as  well  as  incorporate  the  findings  of  our 
report  before  moving  forward  to  create  a  DOD-wide  policy.  Because  DOD 
officials  were  not  able  to  provide  us  a  draft  of  this  proposed  expansion  of 
guidance,  and  they  stated  that  they  are  in  the  earliest  stage  of  a  lengthy 
process,  it  is  unclear  when  DOD  will  issue  department-wide  guidance  for 
advertising.  Further,  it  is  unclear  whether  the  guidance  will  clearly  define 
DOD’s  role  in  oversight,  clarify  all  remaining  issues  related  to  sports 
advertising,  and  provide  direction  to  other  types  of  advertising  such  as 
digital  advertising  and  concerts  or  other  types  of  event  advertising. 

Without  a  policy  that  clearly  defines  DOD’s  role  in  oversight  of  the 
advertising  activities  of  the  military  service  components  and  outlines  the 
procedures  the  components  should  follow  for  all  types  of  advertising 
activities,  DOD  may  not  be  able  to  ensure  that  the  components  are 
carrying  out  other  types  of  advertising  in  a  manner  that  it  considers 
appropriate.  More  broadly,  while  it  may  be  appropriate  for  the  service 
components  to  have  variations  in  advertising  policies  given  that  they 
operate  separate  programs  in  some  instances  and  we  are  encouraged 
that  DOD  officials  stated  that  they  will  address  current  weaknesses  such 
as  those  we  identify  in  this  report,  without  a  policy  that  defines  procedures 
for  all  types  of  advertising  activities,  DOD  risks  abuses  or  inappropriate 
activities  in  other  advertising  in  the  future. 


Conclusions 


Advertising  activities  provide  information  and  seek  to  influence  the  beliefs 
and  understanding  of  potential  recruits  about  each  military  service,  and 
the  services  conduct  advertising  to  help  meet  their  recruitment  goals.  The 
unique  branding  of  each  service  plays  a  role  in  the  decision  of  an 
individual  to  become  a  soldier,  sailor,  airman,  or  marine  and,  as  such,  the 
department  relies  on  the  military  services  to  carry  out  their  own 
advertising  programs.  However,  despite  the  competition  for  potential 
recruits,  a  formal  process  for  coordination  of  advertising  activities  among 
the  military  service  components  could  improve  the  department’s  ability  to 
leverage  resources  and  thus  improve  the  efficiency  of  DOD’s  advertising 
activities.  Further,  while  the  components  generally  follow  commercial  best 
practices  we  identified  for  evaluating  advertising,  DOD  has  not  addressed 
variations  in  measurable  goals  among  the  components  or  insufficient  data 
that  has  prevented  some  components  from  being  able  to  assess  the 
effectiveness  of  their  advertising  activities  in  generating  leads.  Lastly,  the 
absence  of  a  department-wide  policy  that  clearly  defines  DOD  oversight 
of  and  procedures  to  guide  the  advertising  activities  has  allowed  for 
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activities  of  questionable  appropriateness  to  occur  in  some  instances. 
Without  such  a  policy,  DOD  cannot  ensure — through  comprehensive 
oversight — that  each  service  or  component  is  carrying  out  advertising  that 
meets  departmental  standards  and  that  appropriately  invests  taxpayer 
dollars. 


Recommendations  for  recommenc'  ^at  Secretary  of  Defense  take  the  following  three 

actions: 

Executive  Action 

•  Direct  the  Under  Secretary  of  Defense  for  Personnel  and  Readiness, 
in  consultation  with  officials  from  the  military  service  components  and 
the  JAMRS  office,  to  develop  a  formal  process  for  coordination  on 
crosscutting  issues  to  facilitate  better  leveraging  of  resources.  As  part 
of  this  process,  DOD  could  review  existing  advertising  programs  to 
identify  opportunities  to  reduce  unnecessary  duplication,  overlap,  and 
fragmentation  and  obtain  potential  efficiencies. 

•  Direct  the  Secretaries  of  the  Departments  of  the  Army,  the  Navy,  and 
the  Air  Force  to  ensure  that  each  military  service  component  fully 
measure  advertising  performance.  This  should  include  both  the 
identification  of  measurable  goals  in  future  versions  of  the  service 
components’  advertising  plans  and  assurance  that  the  service 
components  have  access  to  the  necessary  performance  data  to 
determine  the  effectiveness  of  their  advertising  activities  for  lead 
generation  activities. 

•  Direct  the  Under  Secretary  of  Defense  for  Personnel  and  Readiness 
to  ensure,  as  the  department  undertakes  its  effort  to  issue  a 
department-wide  policy  for  advertising,  that  this  policy  (1)  clearly 
defines  DOD’s  role  in  overseeing  the  advertising  activities  of  military 
service  components;  (2)  clarifies  issues  related  to  sports-related 
advertising;  and  (3)  outlines  procedures  that  should  guide  the 
components’  advertising  activities  for  other  types  of  advertising,  such 
as  music  concerts,  other  event  advertising,  and  digital  advertising. 


Agency  Comments 
and  Our  Evaluation 


We  provided  a  draft  of  this  report  to  DOD  for  comment.  In  written 
comments,  DOD  generally  concurred  with  our  three  recommendations. 
Specifically,  DOD  concurred  with  our  recommendations  aimed  at 
improving  coordination  and  providing  more  direction  to  military  service 
advertising.  DOD  partially  concurred  with  our  recommendation  regarding 
the  need  for  better  performance  measurement.  DOD  stated  in  its  written 
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comments  that  it  will  work  with  the  military  services  to  develop  guidance 
that  addresses  our  recommendations,  which  will  be  provided  in  the  form 
of  a  DOD  issuance.  DOD’s  comments  are  reprinted  in  their  entirety  in 
appendix  IV.  DOD  also  provided  technical  comments,  which  we 
incorporated  into  the  report  as  appropriate. 

DOD  concurred  with  our  first  recommendation  that  the  department 
develop  a  formal  process  for  coordination  on  crosscutting  issues  to 
facilitate  better  leveraging  of  resources.  In  its  written  comments,  DOD 
stated  that  it  is  developing  a  DOD  instruction  for  marketing  and  that  this 
guidance  will  formalize  coordination  among  the  military  services,  which  it 
states  should  facilitate  better  leveraging  of  resources. 

DOD  partially  concurred  with  our  second  recommendation  that  the 
military  departments  fully  measure  advertising  performance.  We 
recommended  that  this  performance  measurement  should  include  both 
the  identification  of  measurable  goals  in  future  versions  of  the  service 
components’  advertising  plans  and  assurance  that  the  service 
components  have  access  to  the  necessary  performance  data  to 
determine  the  effectiveness  of  their  advertising  activities  for  lead 
generation  activities.  In  its  written  comments,  DOD  stated  that  it  agrees 
with  our  recommendation  in  broad  terms.  DOD  highlighted  actions 
already  underway,  and  stated  that  as  part  of  the  development  of  its 
instruction  for  marketing,  it  will  further  clarify  and  codify  guidance  related 
to  performance  measurement.  However,  DOD  further  states  that  not  all 
goals  and  measures  relate  to  lead  generation  and  that  other  goals  and 
objectives  can  be  used  to  measure  success.  We  acknowledge  in  this 
report  that  the  goal  of  some  forms  of  advertising  is  to  improve  awareness 
or  engagement,  and  not  solely  lead  generation.  Therefore,  we  do  not 
believe  that  all  measurement  and  performance  data  should  be  tied  back 
to  lead  generation  if  that  is  not  the  goal  of  the  advertising  activity  in 
question.  However,  as  we  state  in  our  report,  we  found  that  not  all  military 
services  collect  the  necessary  performance  data  to  determine  if  activities 
intended  to  generate  leads  are  performing  as  intended,  and  DOD  did  not 
address  this  issue  of  data  collection  in  its  written  comments.  We  believe 
that  DOD  and  GAO  largely  agree  on  this  issue  and  are  encouraged  that 
DOD  plans  to  issue  an  instruction  that  will  clarify  the  need  for  better 
performance  measurement  of  advertising  activities.  While  its  September 
2015  interim  guidance  was  an  important  step  that  required  assessment  of 
performance  of  sports  related  advertising,  we  reiterate  that  the  draft  DOD 
instruction  in  development  should  clarify  that  performance  measurement 
is  important  for  all  types  of  advertising,  not  only  sports  related  advertising 
activities,  and  that  the  appropriate  goals,  performance  data,  and 
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performance  measures  be  used  to  assess  the  performance  of  all 
advertising  activities. 

DOD  concurred  with  our  third  recommendation  that  the  department  issue 
policy  for  advertising.  In  its  written  comments,  DOD  stated  that  it  is 
developing  a  DOD  instruction  for  marketing  and  that  this  instruction  will 
clearly  define  DOD’s  role  in  overseeing  the  advertising  activities  of 
military  service  components;  clarify  issues  related  to  sports-related 
advertising;  and  outline  procedures  that  should  guide  the  components’ 
advertising  activities  for  other  types  of  advertising. 


We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees,  the  Secretary  of  Defense;  the  Under  Secretary  of  Defense 
for  Personnel  and  Readiness;  the  Director  of  JAMRS;  and  the  Secretaries 
of  the  Army,  the  Navy,  and  the  Air  Force.  In  addition,  the  report  is 
available  at  no  charge  on  the  GAO  website  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  about  this  report,  please  contact 
Andrew  Von  Ah  at  (213)  830-101 1  or  vonaha@gao.gov.  Contact  points 
for  our  Offices  of  Congressional  Relations  and  Public  Affairs  may  be 
found  on  the  last  page  of  this  report.  GAO  staff  who  made  key 
contributions  to  this  report  are  listed  in  appendix  V. 


Andrew  Von  Ah 

Acting  Director,  Defense  Capabilities  and  Management 
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Appendix  I:  Objectives,  Scope,  and 
Methodology 


The  objectives  of  our  review  were  to  examine  the  extent  to  which  the 
Department  of  Defense  (DOD)  (1)  has  coordinated  its  advertising 
activities  among  the  military  service  components,1  (2)  has  followed 
commercial  best  practices  to  assess  the  effectiveness  of  advertising 
activities,  and  (3)  has  oversight  of  the  components  ‘advertising  activities. 

To  determine  the  extent  to  which  DOD  has  coordinated  its  advertising 
activities  among  the  military  service  components,  we  reviewed 
department-  and  service-level  guidance  pertaining  to  roles  and 
responsibilities  for  advertising.  We  interviewed  officials  from  the 
Accessions  Policy  office  within  the  Office  of  the  Under  Secretary  of 
Defense  for  Personnel  and  Readiness,  the  Joint  Advertising  and 
Marketing  Research  Service  (JAMRS),  and  each  of  the  military  service 
components’  advertising  programs  to  discuss  the  ways  in  which  DOD  has 
taken  steps  to  coordinate  advertising  activities  as  well  as  ways  in  which 
the  military  service  components  coordinate  with  each  other  to  achieve 
efficiencies  in  advertising.  Further,  we  discussed  any  methods  used  to 
share  best  practices  or  lessons  learned  for  advertising,  either  formally  or 
informally,  among  the  military  service  components.  We  also  conducted 
interviews  in  Fort  Knox,  Kentucky,  with  officials  from  the  U.S.  Army 
Recruiting  Command,  the  U.S.  Army  Cadet  Command,  and  the  U.S. 

Army  Accessions  Support  Brigade,  organizations  that  play  a  role  in 
advertising  for  the  active  and  reserve  Army  components.  We  selected 
these  Army  organizations  for  this  objective  because  the  Army  is  the  only 
service  component  that  has  an  office  responsible  for  advertising 
functions — the  Army  Marketing  and  Research  Group — that  is  separate 
from  the  recruiting  function,  carried  out  by  the  U.S.  Army  Recruiting 
Command.  We  compared  identified  policies  and  practices  that  describe 
any  coordination,  as  well  as  any  instances  of  coordination  described  by 
officials,  against  a  best  practice  identified  by  private  sector  advertising 
experts  during  the  course  of  this  review  and  our  duplication,  overlap,  and 
fragmentation  evaluation  and  management  guide.2  Specifically,  the 
private  sector  advertising  experts  we  met  with  during  the  course  of  this 
review  stated  that  it  is  a  best  practice  to  consider  an  effective  use  of 


^he  military  service  components  included  in  this  report  are:  Army  active  duty,  reserve, 
and  Army  National  Guard;  Navy  active  duty  and  reserve;  Marine  Corps  active  duty  and 
reserve;  Air  Force  active  duty,  reserve,  and  Air  National  Guard. 

2GAO,  Fragmentation,  Overlap,  and  Duplication:  An  Evaluation  and  Management  Guide, 
GAO-1 5-49SP  (Washington,  D.C.:  Apr.  14,  2015). 
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coordination  to  increase  efficiencies  in  advertising.  As  there  are  seven 
advertising  programs  carried  out  by  the  military  services  as  well  as 
JAMRS,  we  reviewed  these  programs  against  the  evaluation  and 
management  guide  as  it  describes  how  to  identify  and  evaluate  instances 
of  fragmentation  (more  than  one  agency  involved  in  the  same  broad 
area),  overlap  (multiple  agencies  or  programs  with  similar  goals, 
activities,  or  beneficiaries),  and  duplication  (two  or  more  agencies  or 
programs  engaged  in  the  same  activities  or  services  for  the  same 
beneficiaries)  among  programs.  Further,  the  guide  can  help  identify 
options  to  reduce  or  better  manage  the  negative  effects  of  fragmentation, 
overlap,  and  duplication,  and  evaluate  the  potential  trade-offs  and 
unintended  consequences  of  these  options. 

We  took  four  major  steps  to  determine  the  extent  to  which  DOD  has 
followed  commercial  best  practices  to  assess  the  effectiveness  of 
advertising  activities.  First,  we  interviewed  officials  from  the  Office  of  the 
Under  Secretary  of  Defense  for  Personnel  and  Readiness  and  each  of 
the  advertising  programs  of  the  military  service  components  to  determine 
the  types  of  advertising  they  conduct  and  how  its  effectiveness  is 
determined.  Second,  to  identify  existing  best  practices  for  assessing 
effectiveness  of  advertising,  we  conducted  a  preliminary  search,  including 
a  literature  review,  and  determined  that  a  well-defined  and  widely 
accepted  list  of  best  practices  had  not  been  established.  Therefore,  we 
selected  a  nongeneralizable  sample  of  advertising  companies  and 
professional  organizations  using  a  “snowball  sampling  methodology,” 
which  consisted  of  interviewing  advertising  industry  experts  from  an  initial 
set  of  organizations  and  requesting  those  experts  to  refer  additional 
contacts  to  participate  in  the  review.  Based  on  this  approach,  we 
identified  and  interviewed  the  following  organizations: 

•  Companies:  Ad  Council,  Agent,  WideOpen,  Widmeyer 

•  Professional  organizations:  Association  of  National  Advertisers, 
Advertising  Research  Foundation 

An  executive  from  another  private  sector  advertising  company  preferred 
that  we  not  include  the  name  of  the  organization  that  currently  employs 
the  individual,  stating  that  the  input  provided  during  the  course  of  our 
review  reflected  experience  obtained  from  numerous  positions  held  at 
various  advertising  companies. 

During  the  interviews,  we  asked  officials  to  describe  their  knowledge  of 
industry  best  practices  for  assessing  the  effectiveness  of  advertising. 
Based  on  the  interviews,  we  identified  key  practices  reported  by  each 
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expert  or  organization,  including  organizational  structures  to  safeguard 
against  bias  in  performance  evaluation,  processes  for  planning  and  goal¬ 
setting  in  advance  of  advertising,  and  standard  performance  measures 
used  to  assess  advertising.  We  compiled  these  practices  into  a  list  and 
provided  the  list  to  the  organizations  for  review  and  comment.  Although 
the  perspectives  of  the  organizations  included  in  our  sample  are  not 
generalizable  to  the  advertising  industry  as  a  whole,  we  found  a  sufficient 
degree  of  consensus  among  the  sample  for  the  purpose  of  developing 
criteria  for  this  review. 

Third,  following  the  development  of  the  best  practices,  we  interviewed 
officials  from  each  of  the  military  service  components  responsible  for 
advertising,  as  well  as  the  components’  contracted  advertising  agencies, 
to  determine  the  extent  to  which  the  components  follow  the  practices  we 
identified.  We  compared  the  most  recent  annual  advertising  plan  of  each 
of  the  military  service  components  (which  ranged  from  fiscal  year  2014 
through  2016,  depending  on  the  component)  and  other  department-  and 
service-level  guidance  pertaining  to  advertising  with  the  Standards  for 
Internal  Control  in  the  Federal  Government,  which  requires  that  an 
organization  have  relevant,  reliable,  and  timely  information  and  that  there 
is  communication  of  that  information  throughout  the  agency  in  order  to 
achieve  its  objectives.3  We  also  considered  regulatory  requirements  or 
other  potential  barriers  to  government  agencies  following  the  best 
practices  we  identified. 

Finally,  to  determine  how  performance  data  related  to  advertising 
activities  is  collected  and  reviewed  below  the  headquarters  level,  we 
developed  and  administered  a  structured  questionnaire  to  lower  level 
recruiting  officials  with  responsibility  for  local  advertising  from  two  lower 
level  units  per  service  component.  Based  on  our  discussions  with  the 
components’  headquarters-level  advertising  officials,  we  determined  the 
appropriate  officials  to  participate  in  the  semistructured  interview  were 


3See  GAO,  Standards  for  Internal  Control  in  the  Federal  Government, 
GAO/AIMD-OO-21.3.1  (Washington,  D.C.:  November  1999).  These  standards  were  in 
effect  prior  to  fiscal  year  2016  and  cover  the  time  period  of  most  of  service  components’ 
advertising  plans.  The  standards  were  subsequently  updated.  The  updated  standards 
went  into  effect  on  October  1 , 201 5,  and,  consistent  with  the  old  version  of  the  standards, 
emphasize  the  need  for  relevant,  reliable,  and  timely  information  and  communication  of 
that  information  throughout  the  agency  in  order  to  achieve  all  of  its  objectives.  See  GAO, 
Standards  for  Internal  Control  in  the  Federal  Government,  GAO-14-704G  (Washington, 
D.C.:  Sept.  10,  2014). 
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located  at  the  brigade  level  of  the  Army,  district  level  of  the  Navy  and 
Marine  Corps,  squadron  level  of  the  Air  Force  active  and  reserve 
components,  and  state  level  of  the  Army  and  Air  National  Guards.  Given 
the  total  population  of  177  units,  we  chose  to  use  a  judgmental, 
nonprobability  sampling  approach  to  select  2  subordinate  units  from  each 
of  the  seven  components,  for  a  total  14  interviews.  We  selected  the  units, 
in  consultation  with  service  component  headquarters  officials,  using 
criteria  intended  to  obtain  a  variation  of  perspectives  within  each  of  the 
components.  Due  to  the  degree  of  differences  among  the  components  in 
command  structure  and  methods  for  allocating  advertising  funds  across 
subordinate  units,  we  tailored  the  criteria  to  the  characteristics  of  each 
component.  For  several  components,  we  selected  units  based  on  the  size 
of  the  budget  received  for  local  advertising,  choosing  a  unit  with  a  larger- 
than-average  advertising  budget  and  a  unit  with  a  smaller-than-average 
advertising  budget.  If  a  component  provided  roughly  equal  local 
advertising  funding  to  all  subordinate  units,  we  used  another  selection 
criterion  to  obtain  varying  perspectives,  such  as  selecting  two  units  that 
represented  regions  with  different  propensities  to  serve  or  two  units  with 
different  recruiting  missions  such  as  an  enlisted  unit  and  a  health 
professionals  unit.  We  relied  on  the  headquarters  officials  from  each  of 
the  service  components  to  identify  states  or  units  that  matched  our 
selection  criteria.  Because  we  interviewed  a  nongeneralizable  sample, 
the  results  cannot  be  used  to  make  inferences  about  the  population  as  a 
whole. 

To  determine  the  extent  to  which  DOD  has  oversight  of  its  advertising 
activities,  we  reviewed  (1)  department-level  guidance  on  advertising  and 
recruiting  and  (2)  service-level  guidance  on  the  review  and  approval 
processes  related  to  the  services  components’  advertising  programs.  We 
interviewed  officials  from  the  Under  Secretary  of  Defense  for  Personnel 
and  Readiness  about  the  oversight  provided  to  service  advertising 
programs  and  the  role  of  JAMRS.  In  addition,  to  determine  how  the 
military  service  components  provide  direction  to  and  oversight  of  their 
respective  advertising  programs,  we  interviewed  officials  from  the 
headquarters  of  each  of  the  military  services  components’  advertising 
programs  and  we  developed  and  administered  a  structured  questionnaire 
to  lower  level  recruiting  officials  with  responsibility  for  local  advertising 
from  two  lower  level  units  per  service  component.  We  met  with  officials 
from  a  judgmental,  nonprobability  sample  of  lower  level  units  that 
represent  a  geographic  region  or  area  of  the  United  States,  such  as  a 
recruiting  district,  squadron,  or  brigade  depending  on  the  service.  The 
subordinate  level  units  were  selected  to  obtain  distinct  views  within  each 
component,  based  on  variations  in  the  size  of  budgets,  recruiting  goals,  or 
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the  propensity  of  youth  to  serve  in  the  military  within  the  geographical 
area  of  the  unit.  We  compared  any  departmental  and  service  guidance, 
as  well  as  the  information  obtained  during  our  interviews,  against 
Standards  for  Internal  Control  in  the  Federal  Government,  which  states 
that  an  agency’s  organizational  structure  should  feature  planning, 
directing,  and  controlling  activities  to  ensure  that  goals  and  objectives  of 
the  agency  can  be  achieved.4  Further,  internal  controls  require  that 
policies  be  in  place  to  direct  an  agency’s  activities.5  Lastly,  we  reviewed 
contracting  data  from  fiscal  year  2014  and  some  of  fiscal  year  2015  to 
identify  variations  in  the  types  of  activities  included  in  military  service 
component  advertising  contracts.  We  obtained  contracting  data  from 
fiscal  year  2014  and  some  of  2015  because  this  time  frame  included 
information  before  and  after  the  issuance  of  interim  guidance  related  to 
sports  advertising,  including  the  most  recently  available  data  as  of  March 
2016.  Through  our  discussions  with  relevant  experts  and  our  review  of 
past  work,  we  determined  the  data  to  be  sufficiently  reliable  for  the 
purpose  of  corroborating  the  types  of  activities  included  in  advertising 
contracts.6 

We  conducted  this  performance  audit  from  June  2015  to  May  2016  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 


4See  GAO/AIM D-00-21 .3.1 .  These  standards  were  in  effect  prior  to  fiscal  year  2016  and 
cover  the  time  period  of  most  of  the  service  components’  guidance  regarding  advertising. 
The  standards  were  subsequently  updated.  The  updated  standards  went  into  effect  on 
October  1, 2015,  and,  consistent  with  the  old  version  of  the  standards,  emphasize  agency 
planning,  directing,  and  controlling  activities  to  ensure  goals  and  objectives  can  be 
achieved.  Further,  the  new  version  of  the  internal  control  standards  requires  that  an 
agency’s  activities  be  directed  through  policies.  See  GAO-14-704G. 

5GAO,  Standards  for  Internal  Control  in  the  Federal  Government,  GAO-1 4-703G 
(Washington,  D.C.:  September  2014). 

Contracting  data  were  obtained  from  the  Federal  Procurement  Data  System  -  Next 
Generation  (FPDS-NG),  which  is  a  primary  source  for  federal  procurement  data. 
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Within  the  military  departments,  there  are  seven  military  service 
component  advertising  programs  and  the  organizational  structure  of  each 
program  differs.  Further,  there  is  a  joint  advertising  program — Joint 
Advertising  and  Marketing  Research  Service — that  is  to  provide  some 
advertising  functions  for  the  department. 

Within  all  of  the  military  service  components,  except  for  the  Army  Active 
Duty  and  Reserve  Component  Advertising  Program,  the  remaining  six 
programs’  advertising  function  is  part  of  the  recruiting  function  of  the 
respective  military  service  component.  Each  of  the  military  services  has 
its  own  recruiting  structures  and  organizations,  which  are  responsible  for 
the  military  service’s  recruiting  mission  and  functions.  The  role  of  a 
military  service’s  recruiting  command  is  to  provide  support  to  the 
recruiting  force  and  guidance  for  the  recruitment  and  enlistment  process. 
In  addition,  a  recruiting  command  plays  a  role  in  developing  the  recruiting 
goals.  The  commands  are  structured  similarly  across  the  military  services 
with  some  variation  in  organizational  structure.  The  recruiting  command  is 
the  recruiting  headquarters  for  each  military  service,  with  subordinate 
commands  between  the  headquarters  level  and  recruiting  stations  or 
substations  where  frontline  recruiters  work  to  reach  out  to  prospective 
applicants  and  discuss  the  benefits  of  joining  the  military.  Table  2 
describes  each  of  the  seven  advertising  programs  within  the  military 
departments. 
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Table  2:  Advertising  Programs  within  the  Military  Departments 


The  Department  of  the  Army  Army  Active  Duty  and  Reserve  Components  Advertising  Program.  The  Army  has  a 

combined  program  for  the  Army  active  duty  and  reserves.  Prior  to  2012,  advertising  was  carried 
out  as  part  of  the  Army’s  Accessions  Command.  In  2012  with  the  disestablishment  of  the  Army’s 
Accessions  Command,  the  Army’s  advertising  program  was  reorganized  and  the  Army 
Marketing  and  Research  Group  was  created  within  the  Office  of  the  Assistant  Secretary  of  the 
Army  for  Manpower  and  Reserve  Affairs  to  lead  the  national  advertising  program  of  the  Army 
and  to  manage  the  Army  brand.  This  office  is  separate  from  the  recruiting  function  that  is  the 
responsibility  of  the  U.S.  Army  Recruiting  Command,  which  is  under  the  U.S.  Army  Training  and 
Doctrine  Command.  Local  recruiters  that  carry  out  recruitment  and  related  local  advertising 
efforts  are  part  of  the  U.S.  Army  Recruiting  Command.  The  Army’s  Recruiting  Command  is 
organized  into  recruiting  stations,  battalions,  and  brigades  that  represent  regions  of  the  United 
States  and  is  responsible  for  active  and  reserve  recruiting.  Specifically,  there  are  five  enlisted 
recruiting  brigade  and  a  medical  recruiting  brigade,  which  includes  special  operations  recruiting. 
Each  brigade  commands  up  to  8  recruiting  battalions  within  its  geographic  area  for  a  total  of  44 
battalions.  There  are  more  than  260  recruiting  companies  that  provide  the  tactical  control  of 
approximately  948  recruiting  centers  nationwide. 

Army  National  Guard  Advertising  Program.  The  Army  National  Guard  has  a  separate 
advertising  program  from  the  rest  of  the  Army.  A  headquarters-level  office  leads  the  national 
advertising  and  branding  efforts  and  provides  some  degree  of  coordination,  but  each  state  or 
territory  unit  of  the  Army  National  Guard  is  responsible  for  its  own  state  and  local  advertising 
efforts  to  meet  the  recruiting  and  retention  goals  within  that  state.  There  are  54  state  and 
territory  units  within  the  Army  National  Guard. 

The  Department  of  the  Navy  Navy  Active  Duty  and  Reserve  Components  Advertising  Program.  Prior  to  2005,  the  active 

and  reserve  components  of  the  Navy  each  had  its  own  advertising  program.  In  2005,  the  Navy 
joined  the  two  programs  to  create  the  Navy  Recruiting  Command.  The  recruiting  command  is 
organized  into  two  recruiting  regions,  the  Eastern  and  Western  regions  of  the  United  States. 
Each  region  comprises  13  recruiting  districts  that  represent  geographical  areas  and  the  districts 
comprise  smaller  recruiting  zones  and  individual  recruiting  stations.  Navy  officials  stated  that 
each  of  the  26  recruiting  districts  currently  receives  about  $120,000  a  year  to  fund  local 
advertising. 

Marine  Corps  Active  Duty  and  Reserve  Components  Advertising  Program.  The  advertising 
program  of  the  Marine  Corps  is  located  within  the  Marine  Corps  Recruiting  Command.  The 
recruiting  command  comprises  two  recruiting  regions,  the  eastern  and  western  regions  of  the 
United  States.  There  are  six  geographic  recruiting  districts  within  the  regions.  Marine  Corps 
officials  stated  that  about  1 0  percent  of  the  Marine  Corps’  annual  budget  for  advertising  is 
devoted  to  local  advertising. 


Page  39 


GAO-16-396  DOD  Advertising 


Appendix  II:  Military  Service  Component  and 
Joint  Advertising  Organizations 


The  Department  of  the  Air  Force  Air  Force  Active  Duty  Component  Advertising  Program.  The  Strategic  Marketing  Division 

within  the  Air  Force  Recruiting  Service,  which  is  part  of  the  Air  Education  and  Training 
Command,  carries  out  the  active  duty  advertising  program.  This  office  is  responsible  for  all 
national  level  advertising  programs.  The  recruiting  service  is  comprised  of  three  regional 
recruiting  groups  (Northeastern,  Southeastern,  and  Western).  Each  group  represents  9 
recruiting  squadrons.  Of  the  27  recruiting  squadrons,  24  of  these  recruit  for  geographic  enlisted 
recruiting  while  3  of  the  squadrons  are  specialized,  such  as  the  medical  professional  recruiting 
squadron.  Air  Force  officials  stated  that  each  squadron  receives  about  $50,000  for  local 
advertising  activities  every  year. 

Air  Force  Reserve  Component  Advertising  Program.  The  Air  Force  reserve  component 
advertising  program  is  carried  out  by  the  Advertising  Branch  within  the  Air  Force  Reserve 
Command  Recruiting  Service.  Air  Force  reserve  officials  stated  that  flight  chiefs  with  recruiting 
duties  throughout  the  Air  Force  reserve  are  responsible  for  carrying  out  local  advertising. 
Requests  for  local  advertising  funding  are  reviewed  and  approved  by  the  national  level 
advertising  branch,  meaning  that  funding  occurs  as  needed  as  opposed  to  a  set  amount  of 
funding  being  distributed  at  the  local  level. 

Air  National  Guard  Advertising  Program.  The  Air  National  Guard  Recruiting  and  Retention 
Division  carries  out  the  national  level  advertising  and  reviews  and  approves  requests  for  local- 
level  advertising  funding  received  from  the  54  states’  and  territories’  respective  recruiting  and 
retention  programs. 


Source:  GAO  analysis  of  DOD  information.  |  GAO-16-396 
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Each  military  service  component  receives  an  annual  appropriation  to 
carry  out  operations  and  maintenance  activities,  including  advertising  and 
marketing.  Table  3  shows  the  DOD  reported  amounts  allotted  to  the 
military  service  components’  respective  advertising  and  marketing 
activities  from  each  component’s  annual  operations  and  maintenance 
appropriations  for  fiscal  years  2015  through  2017.  The  Army  has  received 
the  highest  allotments  for  its  advertising  programs.  Specifically,  the  Army 
has  received  about  at  least  two  to  three  times  the  amount  that  the  other 
active-duty  components  have  received. 


Table  3:  DOD  Reported  Annual  Allotments  for  Military  Service  Component  Advertising  Programs  for  (Fiscal  Years  2015 
through  2017) 

Dollars  in  millions 

Component 

2015  Actuals 

2016  Enacted 

2017  Estimate 

Army  active  duty  and  Army  reserve3 

$367.7 

$238.1 

$292.6 

Army  National  Guard 

75.9 

95.4 

76.4 

Air  Force  active  duty 

45.0 

28.3 

52.6 

Air  Force  reserve 

14.4 

7.6 

7.7 

Air  National  Guard 

17.0 

16.6 

16.8 

Navy  active  duty  and  Navy  reserve 

56.1 

49.0 

47.0 

Marine  Corps  active  duty  and  Marine  Corps  reserve 

86.3 

81.5 

81.8 

Total 

$662.4 

$516.5 

$574.9 

Source:  GAO  analysis  of  data  provided  by  Department  of  Defense  Comptroller.  |  GAO-16-396 


aThe  Army  active  duty  and  reserve  annual  allotments  include  amounts  for  some  recruiting  initiatives 
and  for  the  Army’s  Accessions  Support  Brigade. 


In  addition,  each  military  service  component  sets  annual  goals  for 
recruitment  and  retention,  for  both  enlisted  servicemembers  and  officers, 
in  order  to  meet  defined  endstrength  requirements.  Recruits  must  meet 
numerous  standards  before  they  are  accessed  into  the  military, 
specifically  physical,  educational,  and  other  standards  (e.g.,  an 
acceptable  record  of  behavior).  For  the  reserve  and  guard  components, 
the  geographic  location  of  vacancies  is  also  important  for  understanding 
and  addressing  recruitment  and  retention  goals  because  these 
components  recruit  from  a  geographic  area  to  fill  vacancies  in  that  area 
and  recruits  typically  continue  to  live  and  work  in  that  area.  Conversely, 
active  duty  components  recruit  from  anywhere  within  the  United  States 
and  its  territories  to  meet  recruitment  goals  and  recruits  are  assigned 
subsequently  to  duty  stations.  Table  4  shows  the  recruitment  goals  by 
fiscal  year  from  fiscal  year  2014  through  fiscal  year  201 6,  as  of  March  1 , 
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2016.  The  Army  and  the  Army  National  Guard  had  the  largest  recruiting 
missions  over  this  time  period. 

Table  4:  Annual  Enlisted  Accessions  Goals  by  Military  Service  Component  (Fiscal  Years  2014-16) 

Component 

Fiscal  year  2014 

Fiscal  year  2015 

Fiscal  year  2016 

Army  active  duty 

57,000 

59,000 

62,500 

Army  reserve 

29,313 

27,313 

25,900 

Army  National  Guard 

47,900 

46,600 

46,000 

Navy  active  duty 

33,740 

34,990 

33,804 

Navy  reserve 

3,853 

5,293 

7,014 

Marine  Corps  active  duty 

26,000 

29,450 

30,500 

Marine  Corps  reserve 

8,333 

8,052 

5,105 

Air  Force  active  duty 

24,068 

24,087 

30,418 

Air  Force  reserve 

4,875 

5,000 

6,815 

Air  National  Guard 

9,154 

9,312 

10,362 

Total 

244,236 

249,097 

258,418 

Source:  DOD.  |  GAO-16-396 


Page  42 


GAO-16-396  DOD  Advertising 


Appendix  IV:  Comments  from  the 
Department  of  Defense 


MANPOWER  AND 
RESERVE  AFFAIRS 


OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 

1500  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-1500 


Mr.  Andrew  Von  Ah 

Acting  Director,  Defense  Capabilities  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington  DC  2054B 

Dear  Mr.  Von  Ah, 


APR  2  7  2016 


This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  Draft  Report 
GAO- 16-3  96,  “DOD  ADVERTISING:  Better  Coordination,  Performance  Measurement, 
and  Oversight  Needed  to  Help  Meet  Recruitment  Goals,”  dated  April  5,  2016  (GAO 
Code  352046). 

The  Department  is  providing  official  written  comments  for  inclusion  in  the  report. 
The  Department  concurs  with  two  of  the  report’s  recommendations  and  partially  concurs 
with  the  third.  We  will  work  with  the  Military  Services  to  develop  guidance  addressing 
GAO’s  recommendations.  This  guidance  will  be  promulgated  in  a  DoD  issuance. 

The  enclosure  contains  detailed  Departmental  comments  on  each  of  the  three 
recommendations  made  by  GAO.  The  Department  appreciates  the  opportunity  to 
comment  on  the  draft  report. 


Sincerely, 


Stephanie  P.  Miller 
Director,  Accession  Policy 
(Military  Personnel  Policy) 
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GAO  DRAFT  REPORT  DATED  APRIL  5, 2016 
GAO-16-396  (GAO  CODE  352046) 

“DOD  ADVERTISING:  BETTER  COORDINATION,  PERFORMANCE 
MEASUREMENT,  AND  OVERSIGHT  NEEDED  TO  HELP  MEET  RECRUITMENT 

GOALS” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATION 


RECOMMENDATION  #  1 :  The  GAO  recommend  that  the  Secretary  of  Defense  direct  the 
Under  Secretary  of  Defense  for  Personnel  and  Readiness,  in  consultation  with  officials  from  the 
military  service  components  and  the  JAMRS  office,  to  develop  a  formal  process  for  coordination 
on  crosscutting  issues  to  facilitate  better  leveraging  of  resources.  As  part  of  this  process,  DoD 
could  review  existing  advertising  programs  to  identify  opportunities  to  reduce  unnecessary 
duplication,  overlap,  and  fragmentation  and  obtain  potential  efficiencies. 

DoD  RESPONSE:  Concur 

The  Department  agrees  with  the  GAO’s  recommendation.  As  noted  in  the  GAO’s  report,  the 
Department  does  currently  coordinate  advertising  activities  among  the  Services;  such  actions  are 
done  informally.  The  new  Department  of  Defense  Instruction  on  marketing,  currently  under 
development,  will  establish  processes  that  will  formalize  the  coordination  among  the  Services 
which  should  facilitate  better  leveraging  of  resources. 


RECOMMENDATION  #  2: 

The  GAO  recommend  that  the  Secretary  of  Defense  direct  the  Secretaries  of  the  Departments  of 
the  Army,  the  Navy,  and  the  Air  Force  to  ensure  that  each  Military  Service  component  fully 
measures  advertising  performance.  This  should  include  both  the  identification  of  measureable 
goals  in  future  versions  of  the  service  components’  advertising  plans  and  ensuring  that  the 
service  components  have  access  to  the  necessary  performance  data  to  determine  the  effectiveness 
of  their  advertising  activities  for  lead  generation  activities. 

DoD  RESPONSE:  Partially  Concur 

The  Department  agrees  with  this  recommendation  in  broad  terms,  and  believes  each  Military 
Service  component  should  measure  advertising  performance.  Effective  measurements  are 
predicated  on  identifying  measureable  goals  in  the  Service  components’  advertising  plans  and 
having  access  to  the  necessary  performance  data  to  determine  the  effectiveness.  The  Department 
has  already  directed  these  actions  in  the  interim  guidance  provided  to  the  Services  in  a 
Memorandum  signed  by  the  Acting  Undersecretary  of  Defense  for  Personnel  and  Readiness  on 
September  14,  2015.  The  Department  will  further  clarify  and  codify  this  guidance  in  a  new 
proposed  Department  of  Defense  Instruction  on  marketing.  We  do  not  agree  with  the  GAO’s 
implication  that  the  measurable  goals  and  performance  data  should  be  focused  solely  on  lead 
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generation  activities.  Marketing  campaigns  and  engagements  are  designed  and  executed  with 
varying  objectives.  A  campaign  or  engagement  may  not  be  “lead  centric.”  The  Department 
believes  other  measurable  goals  and  performance  data  can  and  should  be  used  to  measure 
success. 


RECOMMENDATION  #  3:  The  GAO  recommend  that  the  Secretary  of  Defense,  as  the 
Department  undertakes  its  effort  to  issue  a  Department-wide  policy  for  advertising,  direct  the 
Under  Secretary  of  Defense  for  Personnel  and  Readiness  to  ensure  the  policy  (1)  clearly  defines 
DoD’s  role  in  overseeing  the  advertising  activities  of  military  service  components,  (2)  clarifies 
issues  related  to  sports  related  advertising,  and  (3)  outlines  procedures  that  should  guide  the 
components’  advertising  activities  for  other  types  of  advertising,  such  as  concerts  or  other  event 
advertising  and  digital  advertising. 

DoD  RESPONSE:  Concur 

The  new  Department  of  Defense  Instruction  regarding  marketing  will  clearly  define  DoD’s  role 
in  overseeing  the  advertising  activities  of  Military  Service  components,  clarify  issues  related  to 
sports  related  advertising,  and  outline  procedures  to  guide  the  components’  advertising  activities 
for  other  types  of  advertising,  such  as  concerts  or  other  event  advertising  and  digital  advertising. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  website  (http://www.gao.gov).  Each  weekday 
afternoon,  GAO  posts  on  its  website  newly  released  reports,  testimony, 
and  correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted 
products,  go  to  http://www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  website, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates. 

Listen  to  our  Podcasts  and  read  The  Watchblog. 

Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 

71 25,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 
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